Skip to contentSkip to conversation

Support new Marvell bootloaders for Armada 370/XP

PATCH New Thomas Petazzoni 3 participants 9 messages May 23, 2013

Subsystems (4)

v2 of 3 New Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover Support new Marvell bootloaders for Armada 370/XP

Hello,

This is the second version of this patch series, that switches Armada 370/XP to use internal registers mapped at 0xf1000000, in order to comply with the new bootloaders shipped by Marvell. It also implements a temporary solution to allow users of platforms having an old bootloader to continue to boot properly. See the detailed explanation below.

Changes since v1
================

* The patch series is now based on jcooper/mvebu/fixes + jcooper/mvebu/cleanup. As a result, I've dropped the first three patches of this series, that have been merged by Jason Cooper already.

* This I'm based on mvebu/fixes + mvebu/cleanup, I had to fix a conflict in "arm: mvebu: move cache and mvebu-mbus initialization later" since that conflicted with the removal of the coherent DMA workaround.

* Improved/fixed two comments in armada-370-xp.c related to the register switch. Noticed and suggested by Andrew Lunn.

* Fixed the virtual address used to remap the internal registers at the new address. It needs to be 2 MB aligned, like the one for the internal registers at the old address, in order to be compatible with what the assembly code in arch/arm/kernel/head.S is doing to set up an early virtual->physical mapping. (2 MB because sections with LPAE are 2 MB wide, and we support LPAE).

In detail, the patch series has
===============================

* Patch 1 and 2 slightly rework the way the SMP initialization and the coherency unit initialization is done, in order to avoid the hardcoded virtual address in the coherency.c file.

To be applied to mvebu/soc

* Patch 3 moves the L2 cache and mvebu-mbus initialization a bit later (i.e from ->init_early() to ->init_time()) so that they can ioremap() their registers without crashing once we remove the static I/O mapping (next patch).

* Patch 4 removes the static I/O mapping that was set up at ->map_io() time to cover the entire "internal registers" area.

To be applied to mvebu/soc

* Patch 5 removes the hardcoded physical address in the SMP initialization code.

To be applied to mvebu/soc

* Patch 6 implements the internal register address change itself. The DTS changes are included in the same patch as the code changes. It is possible to split them in two patches, but applying just one patch and not the other would lead to a kernel that doesn't boot at all.

To be applied to mvebu/soc (even though there are admittedly some DT changes in here).

Additional details
==================

The aim of this patch series is to move the internal registers space of the Armada 370 and Armada XP support to 0xf1000000 instead of 0xd0000000, and prepare for the support of the new versions of U-Boot installed by Marvell on Armada 370 and XP platforms that boot the kernel with the internal registers space already mapped at 0xf1000000.

Before reading the patches, and especially the last one, please read on for a description of the problem and the proposed solution. The solution has been carefully thought of, and all the details explained below are important to understand how the proposed solution was designed.

On Marvell EBU SoCs, including earlier families such as Kirkwood or Dove, all the peripheral registers belong to a memory window of 1 MB, called the "internal registers window". This window of physical memory can be freely moved to an arbitrary physical address. At reset, this memory window is at 0xD0000000, but it can be changed by software to another address if needed.

The *very* important thing to remember is that the mechanism to get or change the base address of this "internal registers" window is a register, which is itself part of this "internal registers" window. The bottom line is that when you run code on a Marvell SoC, you have to *know* where the internal registers are located, because you can't guess dynamically by reading some hardware register. Again, this is a *very* important characteristic, because it explains many of the choices of the proposed solution below.

Historically, the bootloaders for Dove and Kirkwood have always remapped the internal registers window at 0xF1000000, so for those platforms, the Linux kernel has always made the assumption that internal registers are accessible at 0xF1000000.

Unfortunately, for Armada 370 and Armada XP, the current generation of bootloaders that are being shipped on Marvell evaluation boards, but also on products like the Plathome OpenBlocks AX3 or the Globalscale Mirabox do not do the remapping, and are leaving the internal registers at 0xD0000000.

However, this is causing some problems:

1 The internal registers window sits at 3 GB, in the middle of the
0-4 GB area of physical memory, which is quite annoying when you have more than 3 GB of memory. Having them at 0xF1000000 means that you lose less RAM when you have more than 3 GB of physical memory installed. And since Armada XP is LPAE capable, you can even have more than 4 GB memory: we already have boards with 8 GB of memory installed.

2 This is different from Kirkwood and Dove, which makes sharing some
early code like earlyprintk complicated, while our goal is ultimately to merge all Marvell EBU platforms into mach-mvebu.

(1) is really the primary motivation here, (2) is only a nice side-effect.

So, the new generation of bootloaders that are shipped on new Marvell Armada 370/XP platforms are doing the remapping at 0xF1000000 prior to starting Linux. The current kernel cannot boot on such platforms.

One solution is to move the kernel to use 0xF1000000, and assume the bootloader has already done the remapping. This can be one option, but it means that newer kernels would no longer boot on platforms that use an old bootloader. All existing users of Globalscale Mirabox or Plathome OpenBlocks AX3, next time they upgrade their kernel, would have a kernel that simply doesn't boot (and without showing *any* message, because even the address of the UART has changed).

If the only Armada 370/XP platforms in use today had been evaluation boards that are available only to a selected number of people, dropping the compatibility would probably have been acceptable. However, with a large number of Mirabox and OpenBlocks AX3 devices being already shipped, we believe that breaking the compatibility is not an option.

Therefore, the last patch of this series adds some early code in the kernel, at the ->map_io() stage, to switch the internal registers from 0xD0000000 to 0xF1000000 if this has not been done already by the bootloader. As it was explained above, we unfortunately can't read the current base address of the internal register window, so we need a different mechanism to know if the bootloader has done the remapping at 0xF1000000 (new generation bootloader) or has left the internal registers at 0xD0000000 (old generation bootloader). In order to distinguish between those two cases, a CP15 bit is being used. Old bootloaders do not touch this CP15, so it is set to 0. New bootloaders set this CP15 bit to 1, so that the kernel knows that the remapping has already been done. The ->map_io() code looks at this bit to know if the remapping should be done or not.

Unfortunately, tweaking ->map_io() is not sufficient: we also want earlyprintk to work. And earlyprintk is used *before* ->map_io() is called, and *after* ->map_io() is called. So in the earlyprintk code, we also need to do a little bit of magic to know if we're booted on an old or new bootloader, and if the remapping has occured or not in the case of an old bootloader. Since the CP15 bit that we use gets cleared the first time the 'wfi' instruction is cleared, we use this bit only in the early kernel boot. As soon as ->map_io() as done its job, it sets a global variable that tells the earlyprintk code that the registers are now at 0xF1000000.

I hope this long explanation gives enough context to understand the problem and the proposed solution. Do not hesitate to ask for more details if needed.

Notice that the solution would be much simpler if we could do this remapping earlier in the kernel initialization. Unfortunately, ->map_io() is really the first piece of code that is being called into SoC-specific code, so it's the only way of doing this without making changes in generic places in the kernel.

This code has been tested with both an old bootloader and a new bootloader, on the Armada XP GP platform.

Best regards,

Thomas

Thomas Petazzoni (6):
arm: mvebu: remove dependency of SMP init on static I/O mapping
arm: mvebu: avoid hardcoded virtual address in coherency code
arm: mvebu: move cache and mvebu-mbus initialization later
arm: mvebu: remove hardcoded static I/O mapping
arm: mvebu: don't hardcode a physical address in headsmp.S
arm: mvebu: support new bootloaders shipped by Marvell

 arch/arm/boot/dts/armada-370-xp.dtsi |    2 +-
 arch/arm/boot/dts/armada-370.dtsi    |    3 +-
 arch/arm/boot/dts/armada-xp-gp.dts   |    2 +-
 arch/arm/include/debug/mvebu.S       |   77 +++++++++++++++++++--
 arch/arm/mach-mvebu/armada-370-xp.c  |  124 +++++++++++++++++++++++++++++-----
 arch/arm/mach-mvebu/armada-370-xp.h  |    4 +-
 arch/arm/mach-mvebu/coherency.c      |   36 ++++------
 arch/arm/mach-mvebu/coherency.h      |    4 --
 arch/arm/mach-mvebu/common.h         |    2 +
 arch/arm/mach-mvebu/headsmp.S        |   20 +++---
 arch/arm/mach-mvebu/platsmp.c        |   10 ++-
 11 files changed, 220 insertions(+), 64 deletions(-)
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH v2 1/6 arm: mvebu: remove dependency of SMP init on static I/O mapping

The ->smp_init_cpus() function is called very early during boot, at a point where dynamic I/O mappings are not yet possible. However, in the Armada 370/XP implementation of this function, we have to get the number of CPUs. We used to do that by accessing a hardware register, which requires relying on a static I/O mapping set up by ->map_io(). Not only this requires hardcoding a virtual address, but it also prevents us from removing the static I/O mapping.

So this commit changes the way used to get the number of CPUs: we now use the Device Tree, which is a representation of the hardware, and provides us the number of available CPUs. This is also more accurate, because it potentially allows to boot the Linux kernel on only a number of CPUs given by the Device Tree, instead of unconditionally on all CPUs.

As a consequence, the coherency_get_cpu_count() function becomes no longer used, so we remove it.

---
 arch/arm/mach-mvebu/coherency.c |   12 ------------
 arch/arm/mach-mvebu/coherency.h |    4 ----
 arch/arm/mach-mvebu/common.h    |    2 ++
 arch/arm/mach-mvebu/platsmp.c   |   10 +++++++++-
 4 files changed, 11 insertions(+), 17 deletions(-)
arch/arm/mach-mvebu/coherency.c +0 -12
index 8278960..46d66c0 100644
@@ -47,18 +47,6 @@ static struct of_device_id of_coherency_table[] = {
 	{ /* end of list */ },
 };
 
-#ifdef CONFIG_SMP
-int coherency_get_cpu_count(void)
-{
-	int reg, cnt;
-
-	reg = readl(coherency_base + COHERENCY_FABRIC_CFG_OFFSET);
-	cnt = (reg & 0xF) + 1;
-
-	return cnt;
-}
-#endif
-
 /* Function defined in coherency_ll.S */
 int ll_set_cpu_coherent(void __iomem *base_addr, unsigned int hw_cpu_id);
 
arch/arm/mach-mvebu/coherency.h +0 -4
index 2f42813..df33ad8 100644
@@ -14,10 +14,6 @@
 #ifndef __MACH_370_XP_COHERENCY_H
 #define __MACH_370_XP_COHERENCY_H
 
-#ifdef CONFIG_SMP
-int coherency_get_cpu_count(void);
-#endif
-
 int set_cpu_coherent(int cpu_id, int smp_group_id);
 int coherency_init(void);
 
arch/arm/mach-mvebu/common.h +2 -0
index aa27bc2..98defd5 100644
@@ -15,6 +15,8 @@
 #ifndef __ARCH_MVEBU_COMMON_H
 #define __ARCH_MVEBU_COMMON_H
 
+#define ARMADA_XP_MAX_CPUS 4
+
 void mvebu_restart(char mode, const char *cmd);
 
 void armada_370_xp_init_irq(void);
arch/arm/mach-mvebu/platsmp.c +9 -1
index 875ea74..93f2f3a 100644
@@ -88,8 +88,16 @@ static int __cpuinit armada_xp_boot_secondary(unsigned int cpu,
 
 static void __init armada_xp_smp_init_cpus(void)
 {
+	struct device_node *np;
 	unsigned int i, ncores;
-	ncores = coherency_get_cpu_count();
+
+	np = of_find_node_by_name(NULL, "cpus");
+	if (!np)
+		panic("No 'cpus' node found\n");
+
+	ncores = of_get_child_count(np);
+	if (ncores == 0 || ncores > ARMADA_XP_MAX_CPUS)
+		panic("Invalid number of CPUs in DT\n");
 
 	/* Limit possible CPUs to defconfig */
 	if (ncores > nr_cpu_ids) {
Signature · 1 line
1.7.9.5
#2 lore raw

PATCH v2 2/6 arm: mvebu: avoid hardcoded virtual address in coherency code

In reply to Thomas Petazzoni

Now that the coherency_get_cpu_count() function no longer requires a very early mapping of the coherency unit registers, we can avoid the hardcoded virtual address in coherency.c. However, the coherency features are still used quite early, so we need to do the of_iomap() early enough, at the ->init_timer() level, so we have the call of coherency_init() at this point.

Unfortunately, at ->init_timer() time, it is not possible to register a bus notifier, so we add a separate coherency_late_init() function that gets called as as postcore_initcall(), when bus notifiers are available.

---
 arch/arm/mach-mvebu/armada-370-xp.c |    2 +-
 arch/arm/mach-mvebu/coherency.c     |   20 ++++++++++----------
 2 files changed, 11 insertions(+), 11 deletions(-)
arch/arm/mach-mvebu/armada-370-xp.c +1 -1
index cf8e357..b9319c4 100644
@@ -46,6 +46,7 @@ static void __init armada_370_xp_timer_and_clk_init(void)
 {
 	mvebu_clocks_init();
 	armada_370_xp_timer_init();
+	coherency_init();
 }
 
 static void __init armada_370_xp_init_early(void)
@@ -75,7 +76,6 @@ static void __init armada_370_xp_init_early(void)
 static void __init armada_370_xp_dt_init(void)
 {
 	of_platform_populate(NULL, of_default_bus_match_table, NULL, NULL);
-	coherency_init();
 }
 
 static const char * const armada_370_xp_dt_compat[] = {
arch/arm/mach-mvebu/coherency.c +10 -10
index 46d66c0..d74794a 100644
@@ -27,14 +27,7 @@
 #include <asm/smp_plat.h>
 #include "armada-370-xp.h"
 
-/*
- * Some functions in this file are called very early during SMP
- * initialization. At that time the device tree framework is not yet
- * ready, and it is not possible to get the register address to
- * ioremap it. That's why the pointer below is given with an initial
- * value matching its virtual mapping
- */
-static void __iomem *coherency_base = ARMADA_370_XP_REGS_VIRT_BASE + 0x20200;
+static void __iomem *coherency_base;
 static void __iomem *coherency_cpu_base;
 
 /* Coherency fabric registers */
@@ -135,9 +128,16 @@ int __init coherency_init(void)
 		coherency_base = of_iomap(np, 0);
 		coherency_cpu_base = of_iomap(np, 1);
 		set_cpu_coherent(cpu_logical_map(smp_processor_id()), 0);
-		bus_register_notifier(&platform_bus_type,
-					&mvebu_hwcc_platform_nb);
 	}
 
 	return 0;
 }
+
+static int __init coherency_late_init(void)
+{
+	bus_register_notifier(&platform_bus_type,
+			      &mvebu_hwcc_platform_nb);
+	return 0;
+}
+
+postcore_initcall(coherency_late_init);
Signature · 1 line
1.7.9.5
#3 lore raw

PATCH v2 3/6 arm: mvebu: move cache and mvebu-mbus initialization later

In reply to Thomas Petazzoni

Current, the L2 cache and the mvebu-mbus drivers are initialized at ->init_early() time. However, at ->init_early() time, ioremap() only works if a static I/O mapping has already been put in place. If it's not the case, it tries to do a memory allocation with kmalloc() which is not possible so early at this stage of the initialization.

Since we want to get rid of the static I/O mapping, we cannot initialize the L2 cache driver and the mvebu-mbus driver so early. So, we move their initialization to the ->init_time() level, which is slightly later (so ioremap() works properly), but sufficiently early to be before the call of the ->smp_prepare_cpus() hook, which creates an address decoding window for the BootROM, which requires the mvebu-mbus driver to be properly initialized.

---
 arch/arm/mach-mvebu/armada-370-xp.c |    8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)
arch/arm/mach-mvebu/armada-370-xp.c +2 -6
index b9319c4..75ebf56 100644
@@ -44,14 +44,11 @@ static void __init armada_370_xp_map_io(void)
 
 static void __init armada_370_xp_timer_and_clk_init(void)
 {
+	char *mbus_soc_name;
+
 	mvebu_clocks_init();
 	armada_370_xp_timer_init();
 	coherency_init();
-}
-
-static void __init armada_370_xp_init_early(void)
-{
-	char *mbus_soc_name;
 
 	/*
 	 * This initialization will be replaced by a DT-based
@@ -87,7 +84,6 @@ DT_MACHINE_START(ARMADA_XP_DT, "Marvell Armada 370/XP (Device Tree)")
 	.smp		= smp_ops(armada_xp_smp_ops),
 	.init_machine	= armada_370_xp_dt_init,
 	.map_io		= armada_370_xp_map_io,
-	.init_early	= armada_370_xp_init_early,
 	.init_time	= armada_370_xp_timer_and_clk_init,
 	.restart	= mvebu_restart,
 	.dt_compat	= armada_370_xp_dt_compat,
Signature · 1 line
1.7.9.5
#4 lore raw

PATCH v2 4/6 arm: mvebu: remove hardcoded static I/O mapping

In reply to Thomas Petazzoni

Now that we have removed the need of the static I/O mapping for early initialization reasons, and fixed the registers area length that were broken, we can get rid of the static I/O mapping. Only the earlyprintk mapping needs to be set up, using the debug_ll_io_init() helper function.

---
 arch/arm/mach-mvebu/armada-370-xp.c |   11 +----------
 arch/arm/mach-mvebu/armada-370-xp.h |    2 --
 2 files changed, 1 insertion(+), 12 deletions(-)
arch/arm/mach-mvebu/armada-370-xp.c +1 -10
index 75ebf56..c1c0556 100644
@@ -28,18 +28,9 @@
 #include "common.h"
 #include "coherency.h"
 
-static struct map_desc armada_370_xp_io_desc[] __initdata = {
-	{
-		.virtual	= (unsigned long) ARMADA_370_XP_REGS_VIRT_BASE,
-		.pfn		= __phys_to_pfn(ARMADA_370_XP_REGS_PHYS_BASE),
-		.length		= ARMADA_370_XP_REGS_SIZE,
-		.type		= MT_DEVICE,
-	},
-};
-
 static void __init armada_370_xp_map_io(void)
 {
-	iotable_init(armada_370_xp_io_desc, ARRAY_SIZE(armada_370_xp_io_desc));
+	debug_ll_io_init();
 }
 
 static void __init armada_370_xp_timer_and_clk_init(void)
arch/arm/mach-mvebu/armada-370-xp.h +0 -2
index 2070e1b..585e147 100644
@@ -16,8 +16,6 @@
 #define __MACH_ARMADA_370_XP_H
 
 #define ARMADA_370_XP_REGS_PHYS_BASE	0xd0000000
-#define ARMADA_370_XP_REGS_VIRT_BASE	IOMEM(0xfec00000)
-#define ARMADA_370_XP_REGS_SIZE		SZ_1M
 
 /* These defines can go away once mvebu-mbus has a DT binding */
 #define ARMADA_370_XP_MBUS_WINS_BASE    (ARMADA_370_XP_REGS_PHYS_BASE + 0x20000)
Signature · 1 line
1.7.9.5
#5 lore raw

PATCH v2 5/6 arm: mvebu: don't hardcode a physical address in headsmp.S

In reply to Thomas Petazzoni

Now that the coherency_init() function is called a bit earlier, we can actually read the physical address of the coherency unit registers from the Device Tree, and communicate that to the headsmp.S code, which avoids hardcoding a physical address.

---
 arch/arm/mach-mvebu/coherency.c |    4 ++++
 arch/arm/mach-mvebu/headsmp.S   |   20 +++++++++++---------
 2 files changed, 15 insertions(+), 9 deletions(-)
arch/arm/mach-mvebu/coherency.c +4 -0
index d74794a..3d8f40f 100644
@@ -27,6 +27,7 @@
 #include <asm/smp_plat.h>
 #include "armada-370-xp.h"
 
+unsigned long __cpuinitdata coherency_phys_base;
 static void __iomem *coherency_base;
 static void __iomem *coherency_cpu_base;
 
@@ -124,7 +125,10 @@ int __init coherency_init(void)
 
 	np = of_find_matching_node(NULL, of_coherency_table);
 	if (np) {
+		struct resource res;
 		pr_info("Initializing Coherency fabric\n");
+		of_address_to_resource(np, 0, &res);
+		coherency_phys_base = res.start;
 		coherency_base = of_iomap(np, 0);
 		coherency_cpu_base = of_iomap(np, 1);
 		set_cpu_coherent(cpu_logical_map(smp_processor_id()), 0);
arch/arm/mach-mvebu/headsmp.S +11 -9
index a06e0ed..095b590 100644
@@ -21,12 +21,6 @@
 #include <linux/linkage.h>
 #include <linux/init.h>
 
-/*
- * At this stage the secondary CPUs don't have acces yet to the MMU, so
- * we have to provide physical addresses
- */
-#define ARMADA_XP_CFB_BASE	     0xD0020200
-
 	__CPUINIT
 
 /*
@@ -35,15 +29,23 @@
  * startup
  */
 ENTRY(armada_xp_secondary_startup)
+	/* Add CPU to coherency fabric */
+	adr	r0, 1f
+	ldmia	r0, {r1, r2}
+	sub	r0, r0, r1
+	add	r2, r2, r0
+	ldr	r0, [r2]
 
 	/* Read CPU id */
 	mrc     p15, 0, r1, c0, c0, 5
 	and     r1, r1, #0xF
 
-	/* Add CPU to coherency fabric */
-	ldr     r0, =ARMADA_XP_CFB_BASE
-
 	bl	ll_set_cpu_coherent
 	b	secondary_startup
 
 ENDPROC(armada_xp_secondary_startup)
+
+	.align 2
+1:
+	.long	.
+	.long	coherency_phys_base
Signature · 1 line
1.7.9.5
#6 lore raw

PATCH v2 6/6 arm: mvebu: support new bootloaders shipped by Marvell

In reply to Thomas Petazzoni

On Marvell platforms, the "internal registers" is a window of 1 MB of the physical address space that contains all the registers for the various peripherals of the SoC (timers, interrupt controllers, GPIO controllers, Ethernet, etc.). The base address of this 1 MB window is configurable, and this base address is configured using a register that is itself part of this window.

All earlier families of Marvell EBU SoCs supported by Linux used 0xf1000000 as the base address of the internal registers window. For some reason, early versions of the Marvell 370/XP platforms used 0xd0000000 as the base address. This base address is set up by the bootloader. However, in order to keep as much physical address space to address RAM below the 4 GB limit, Marvell had to switch the base address of the internal registers window back to 0xf1000000.

All the new bootloaders shipped by Marvell remap those registers to 0xf1000000. This commit adapts the kernel Device Tree and other definitions so that it can boot with such bootloaders.

However, since those changing are breaking the compatibility with old bootloader (resulting in a kernel that crashes without displaying any message), we have implement a temporary solution for users of an old bootloader.

One particular bit of a CP15 register has been chosen to indicate whether we use an old bootloader (setting the address at 0xd0000000) or a new bootloader (setting the address at 0xf1000000). In the ->map_io() function, when the bit is already set, there is nothing to do, we are already using a new bootloader and internal registers are at 0xf1000000.

However, when the bit is not set, we do a static mapping of the register area that allows to change the internal register address, we change this address, and set this bit.

Note that this mechanism also needs cooperation from the earlyprintk addruart handler (in arch/arm/include/debug/mvebu.S). This is where things get complex: the CP15 bit that has been chosen gets reset to zero at the first call to the 'wfi' instruction. So it cannot be used during the entire life of the system. Instead, whenever the ->map_io() function has switched to the internal register space to the new address, it sets a global variable 'mvebu_switched_regs', which the earlyprintk reads. If it's true, then the move has already been done and there is no need to read the CP15 register. If 'mvebu_switched_regs' is false, we are before the register move, so we need to read the CP15 bit to find out whether we're being booted from an old bootloader (which mapped the register space at 0xd0000000) or a new bootloader (which mapped the register space at 0xf1000000).

---
 arch/arm/boot/dts/armada-370-xp.dtsi |    2 +-
 arch/arm/boot/dts/armada-370.dtsi    |    3 +-
 arch/arm/boot/dts/armada-xp-gp.dts   |    2 +-
 arch/arm/include/debug/mvebu.S       |   77 +++++++++++++++++++++++--
 arch/arm/mach-mvebu/armada-370-xp.c  |  105 ++++++++++++++++++++++++++++++++++
 arch/arm/mach-mvebu/armada-370-xp.h  |    2 +-
 6 files changed, 181 insertions(+), 10 deletions(-)
arch/arm/boot/dts/armada-370-xp.dtsi +1 -1
index 52a1f5e..95ec049 100644
@@ -33,7 +33,7 @@
 		#size-cells = <1>;
 		compatible = "simple-bus";
 		interrupt-parent = <&mpic>;
-		ranges = <0          0 0xd0000000 0x0100000 /* internal registers */
+		ranges = <0          0 0xf1000000 0x0100000 /* internal registers */
 			  0xe0000000 0 0xe0000000 0x8100000 /* PCIe */>;
 
 		internal-regs {
arch/arm/boot/dts/armada-370.dtsi +2 -1
index aee2b18..030e425 100644
@@ -29,8 +29,9 @@
 	};
 
 	soc {
-		ranges = <0          0xd0000000 0x0100000 /* internal registers */
+		ranges = <0          0xf1000000 0x0100000 /* internal registers */
 			  0xe0000000 0xe0000000 0x8100000 /* PCIe */>;
+
 		internal-regs {
 			system-controller at 18200 {
 				compatible = "marvell,armada-370-xp-system-controller";
arch/arm/boot/dts/armada-xp-gp.dts +1 -1
index 3ee63d1..8fb1679 100644
@@ -39,7 +39,7 @@
 	};
 
 	soc {
-		ranges = <0          0 0xd0000000 0x100000
+		ranges = <0          0 0xf1000000 0x100000
 			  0xf0000000 0 0xf0000000 0x1000000>;
 
 		internal-regs {
arch/arm/include/debug/mvebu.S +71 -6

91 lines not on this page · raw message

arch/arm/mach-mvebu/armada-370-xp.c +105 -0

113 lines not on this page · raw message

arch/arm/mach-mvebu/armada-370-xp.h +1 -1
index 585e147..ff9adc4 100644
@@ -15,7 +15,7 @@
 #ifndef __MACH_ARMADA_370_XP_H
 #define __MACH_ARMADA_370_XP_H
 
-#define ARMADA_370_XP_REGS_PHYS_BASE	0xd0000000
+#define ARMADA_370_XP_REGS_PHYS_BASE	0xf1000000
 
 /* These defines can go away once mvebu-mbus has a DT binding */
 #define ARMADA_370_XP_MBUS_WINS_BASE    (ARMADA_370_XP_REGS_PHYS_BASE + 0x20000)
Signature · 1 line
1.7.9.5
maintainer
#7 lore raw

PATCH v2 6/6 arm: mvebu: support new bootloaders shipped by Marvell

Quoted 49 lines · Thomas Petazzoni

On Marvell platforms, the "internal registers" is a window of 1 MB of the physical address space that contains all the registers for the various peripherals of the SoC (timers, interrupt controllers, GPIO controllers, Ethernet, etc.). The base address of this 1 MB window is configurable, and this base address is configured using a register that is itself part of this window.

All earlier families of Marvell EBU SoCs supported by Linux used 0xf1000000 as the base address of the internal registers window. For some reason, early versions of the Marvell 370/XP platforms used 0xd0000000 as the base address. This base address is set up by the bootloader. However, in order to keep as much physical address space to address RAM below the 4 GB limit, Marvell had to switch the base address of the internal registers window back to 0xf1000000.

All the new bootloaders shipped by Marvell remap those registers to 0xf1000000. This commit adapts the kernel Device Tree and other definitions so that it can boot with such bootloaders.

However, since those changing are breaking the compatibility with old bootloader (resulting in a kernel that crashes without displaying any message), we have implement a temporary solution for users of an old bootloader.

One particular bit of a CP15 register has been chosen to indicate whether we use an old bootloader (setting the address at 0xd0000000) or a new bootloader (setting the address at 0xf1000000). In the ->map_io() function, when the bit is already set, there is nothing to do, we are already using a new bootloader and internal registers are at 0xf1000000.

However, when the bit is not set, we do a static mapping of the register area that allows to change the internal register address, we change this address, and set this bit.

Note that this mechanism also needs cooperation from the earlyprintk addruart handler (in arch/arm/include/debug/mvebu.S). This is where things get complex: the CP15 bit that has been chosen gets reset to zero at the first call to the 'wfi' instruction. So it cannot be used during the entire life of the system. Instead, whenever the ->map_io() function has switched to the internal register space to the new address, it sets a global variable 'mvebu_switched_regs', which the earlyprintk reads. If it's true, then the move has already been done and there is no need to read the CP15 register. If 'mvebu_switched_regs' is false, we are before the register move, so we need to read the CP15 bit to find out whether we're being booted from an old bootloader (which mapped the register space at 0xd0000000) or a new bootloader (which mapped the register space at 0xf1000000).

Quoted 1 line

Hi Thomas

Thanks for making the small changes i requested.

Andrew

Quoted 280 lines
---
 arch/arm/boot/dts/armada-370-xp.dtsi |    2 +-
 arch/arm/boot/dts/armada-370.dtsi    |    3 +-
 arch/arm/boot/dts/armada-xp-gp.dts   |    2 +-
 arch/arm/include/debug/mvebu.S       |   77 +++++++++++++++++++++++--
 arch/arm/mach-mvebu/armada-370-xp.c  |  105 ++++++++++++++++++++++++++++++++++
 arch/arm/mach-mvebu/armada-370-xp.h  |    2 +-
 6 files changed, 181 insertions(+), 10 deletions(-)
arch/arm/boot/dts/armada-370-xp.dtsi +1 -1
index 52a1f5e..95ec049 100644
@@ -33,7 +33,7 @@
 		#size-cells = <1>;
 		compatible = "simple-bus";
 		interrupt-parent = <&mpic>;
-		ranges = <0          0 0xd0000000 0x0100000 /* internal registers */
+		ranges = <0          0 0xf1000000 0x0100000 /* internal registers */
 			  0xe0000000 0 0xe0000000 0x8100000 /* PCIe */>;
 
 		internal-regs {
arch/arm/boot/dts/armada-370.dtsi +2 -1
index aee2b18..030e425 100644
@@ -29,8 +29,9 @@
 	};
 
 	soc {
-		ranges = <0          0xd0000000 0x0100000 /* internal registers */
+		ranges = <0          0xf1000000 0x0100000 /* internal registers */
 			  0xe0000000 0xe0000000 0x8100000 /* PCIe */>;
+
 		internal-regs {
 			system-controller at 18200 {
 				compatible = "marvell,armada-370-xp-system-controller";
arch/arm/boot/dts/armada-xp-gp.dts +1 -1
index 3ee63d1..8fb1679 100644
@@ -39,7 +39,7 @@
 	};
 
 	soc {
-		ranges = <0          0 0xd0000000 0x100000
+		ranges = <0          0 0xf1000000 0x100000
 			  0xf0000000 0 0xf0000000 0x1000000>;
 
 		internal-regs {
arch/arm/include/debug/mvebu.S +71 -6
index df191af..ec875c4 100644
@@ -5,20 +5,85 @@
  *
  * Lior Amsalem <[email protected]>
  * Gregory Clement <[email protected]>
+ * Thomas Petazzoni <[email protected]>
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License version 2 as
  * published by the Free Software Foundation.
 */
 
-#define ARMADA_370_XP_REGS_PHYS_BASE	0xd0000000
-#define ARMADA_370_XP_REGS_VIRT_BASE	0xfec00000
+#define ARMADA_370_XP_UART_PHYS_BASE_OLD	0xd0012000
+#define ARMADA_370_XP_UART_PHYS_BASE_NEW	0xf1012000
+#define ARMADA_370_XP_UART_VIRT_BASE_OLD	0xfec12000
+#define ARMADA_370_XP_UART_VIRT_BASE_NEW	0xfea12000
 
 	.macro	addruart, rp, rv, tmp
-	ldr	\rp, =ARMADA_370_XP_REGS_PHYS_BASE
-	ldr	\rv, =ARMADA_370_XP_REGS_VIRT_BASE
-	orr	\rp, \rp, #0x00012000
-	orr	\rv, \rv, #0x00012000
+
+#ifndef ZIMAGE
+	/*
+	 * This loads the value of mvebu_switched_regs. We need a
+	 * little bit of gymnastic here to handle the fact that the
+	 * addruart code is duplicated in different places, and called
+	 * sometimes with the MMU disabled, sometimes the MMU enabled.
+	 *
+	 * Notice that when this earlyprintk code is linked into the
+	 * kernel decompressor (i.e ZIMAGE is defined), this code isn't
+	 * compiled. The reason is because the global mvebu_switched_regs
+	 * variable is not available in the decompressor code. At this
+	 * early stage, we simply test the CP15 bit (below) to know where
+	 * the internal registers are.
+	 */
+	adr	\rp, addruart_switched_regs_\@
+	ldr	\rv, [\rp]
+	sub	\rv, \rv, \rp
+	ldr	\rp, [\rp, #4]
+	sub	\tmp, \rp, \rv
+	ldr	\rp, [\tmp, #0]
+	cmp	\rp, #0
+
+	/* If we have already switched, then use the new register
+	   address */
+	bne	addruart_new_\@
+#endif
+
+	/*
+	 * We haven't switched the register location, so test the CP15
+	 * register bit to find whether we are:
+	 * - with an old bootloader setting the base address to
+	 *   0xd0000000
+	 * - with a new bootloader that has already set the
+	 *   base address to 0xf1000000
+	 * We adapt the physical address returned depending on the
+	 * value of this bit, but also the virtual address, because we
+	 * can't remap the old physical address and the new physical
+	 * address at the same location.
+	 */
+	mrc	p15, 0, \tmp, c5, c0, 0
+	and	\tmp, \tmp, #(1 << 11)
+	cmp	\tmp, #0
+	bne	addruart_new_\@
+
+addruart_old_\@:
+	ldr	\rp, =ARMADA_370_XP_UART_PHYS_BASE_OLD
+	ldr	\rv, =ARMADA_370_XP_UART_VIRT_BASE_OLD
+	b	addruart_out_\@
+addruart_new_\@:
+	ldr	\rp, =ARMADA_370_XP_UART_PHYS_BASE_NEW
+	ldr	\rv, =ARMADA_370_XP_UART_VIRT_BASE_NEW
+	b	addruart_out_\@
+
+#ifndef ZIMAGE
+	/*
+	 * Global variable mvebu_switched_regs not visible in the
+	 * decompressor code
+	 */
+	.align
+addruart_switched_regs_\@:
+	.long	.
+	.long	mvebu_switched_regs
+#endif
+
+addruart_out_\@:
 	.endm
 
 #define UART_SHIFT	2
arch/arm/mach-mvebu/armada-370-xp.c +105 -0
index c1c0556..752eedd 100644
@@ -28,8 +28,113 @@
 #include "common.h"
 #include "coherency.h"
 
+/*
+ * Note on the internal register address.
+ *
+ * On Marvell platforms, the "internal registers" is a window of 1 MB
+ * of the physical address space that contains all the registers for
+ * the various peripherals of the SoC (timers, interrupt controllers,
+ * GPIO controllers, Ethernet, etc.). The base address of this 1 MB
+ * window is configurable, and this base address is configured using a
+ * register that is itself part of this window.
+ *
+ * All earlier families of Marvell EBU SoCs supported by Linux used
+ * 0xf1000000 as the base address of the internal registers
+ * window. For some reason, early versions of the Marvell 370/XP
+ * platforms used 0xd0000000 as the base address. This base address is
+ * set up by the bootloader. However, in order to keep as much
+ * physical address space to address RAM below the 4 GB limit, we had
+ * to switch the base address of the internal registers window back to
+ * 0xf1000000, and we wanted to achieve that without breaking existing
+ * platforms, where the bootloader was initially setting the base
+ * address to 0xd0000000. So the kernel is responsible for switching
+ * to 0xf1000000 when needed.
+ *
+ * One particular bit of a CP15 register has been chosen to indicate
+ * whether we use an old bootloader (setting the address at
+ * 0xd0000000) or a new bootloader (setting the address at
+ * 0xf1000000). In the ->map_io() function, when the bit is already
+ * set, there is nothing to do, we are already using a new bootloader
+ * and internal registers are at 0xf1000000.
+ *
+ * However, when the bit is not set, we do a static mapping of the
+ * register area that allows to change the internal register address
+ * and we change this address.
+ *
+ * Once those steps are done, we set a global variable,
+ * mvebu_switched_regs, that tells the earlyprintk code how to
+ * behave. Note that this mechanism also needs cooperation from the
+ * earlyprintk addruart handler (in arch/arm/include/debug/mvebu.S).
+ */
+
+#define NEW_INTERNAL_REGS_ADDR  0xf1000000
+#define OLD_INTERNAL_REGS_ADDR  0xd0000000
+#define MBUS_OLD_PHYS_ADDR      (OLD_INTERNAL_REGS_ADDR + 0x20000)
+#define MBUS_OLD_VIRT_ADDR      0xfe9ff000
+#define  MBUS_INTERNAL_REGS_ADDR      0x80
+
+/*
+ * We really want this variable to be part of the .data section,
+ * because it gets accessed by the earlyprintk code before BSS gets
+ * initialized to zero. This variable indicates whether the switch to
+ * the new register has taken place or not. It is needed because the
+ * CP15 bit used by the bootloader to tell the kernel where the
+ * registers are mapped is cleared to 0 at the first 'wfi'
+ * instruction. So we can't use it for the entire life of the system,
+ * and we instead use this boolean to indicate to the earlyprintk code
+ * that the switch has occured and it doesn't need to look at the CP15
+ * bit anymore.
+ */
+unsigned int __section(.data) mvebu_switched_regs = 0;
+
+/*
+ * Detect if we're running an old bootloader that has set the physical
+ * base address of internal registers to 0xd0000000
+ */
+static int armada_370_xp_has_old_bootloader(void)
+{
+	u32 val;
+	asm volatile("mrc p15, 0, %0, c5, c0, 0" : "=r"(val));
+	return !(val & BIT(11));
+}
+
 static void __init armada_370_xp_map_io(void)
 {
+	if (armada_370_xp_has_old_bootloader()) {
+		struct map_desc desc;
+		void __iomem *mbus = IOMEM(MBUS_OLD_VIRT_ADDR);
+
+		desc.virtual = MBUS_OLD_VIRT_ADDR;
+		desc.pfn     = __phys_to_pfn(MBUS_OLD_PHYS_ADDR);
+		desc.length  = PAGE_SIZE;
+		desc.type    = MT_DEVICE;
+
+		/*
+		 * Map the area that contains the registers that
+		 * allows to change the base address of internal
+		 * registers.
+		 */
+		iotable_init(&desc, 1);
+
+		/*
+		 * Change the physical base address of the
+		 * registers. From now on, and until
+		 * debug_ll_io_init() is called below, it is not
+		 * possible to do any print, because using earlyprintk
+		 * would lead to memory writes to the old internal
+		 * registers, which would hang the CPU.
+		 */
+		writel(NEW_INTERNAL_REGS_ADDR, mbus + MBUS_INTERNAL_REGS_ADDR);
+	}
+
+	/*
+	 * Either the registers have been switched by the above code,
+	 * or the bootloader had already set the register space at the
+	 * right location. In both cases, tell the earlyprintk code it
+	 * can now use the new location.
+	 */
+	mvebu_switched_regs = 1;
+
 	debug_ll_io_init();
 }
 
arch/arm/mach-mvebu/armada-370-xp.h +1 -1
index 585e147..ff9adc4 100644
@@ -15,7 +15,7 @@
 #ifndef __MACH_ARMADA_370_XP_H
 #define __MACH_ARMADA_370_XP_H
 
-#define ARMADA_370_XP_REGS_PHYS_BASE	0xd0000000
+#define ARMADA_370_XP_REGS_PHYS_BASE	0xf1000000
 
 /* These defines can go away once mvebu-mbus has a DT binding */
 #define ARMADA_370_XP_MBUS_WINS_BASE    (ARMADA_370_XP_REGS_PHYS_BASE + 0x20000)
Signature · 1 line
1.7.9.5
#8 lore raw

v2 cover Support new Marvell bootloaders for Armada 370/XP

In reply to Thomas Petazzoni

Hi Thomas,
On 05/23/2013 03:30 PM, Thomas Petazzoni wrote:

[...]

Quoted 11 lines

Best regards,

Thomas

Thomas Petazzoni (6):
arm: mvebu: remove dependency of SMP init on static I/O mapping
arm: mvebu: avoid hardcoded virtual address in coherency code
arm: mvebu: move cache and mvebu-mbus initialization later
arm: mvebu: remove hardcoded static I/O mapping
arm: mvebu: don't hardcode a physical address in headsmp.S
arm: mvebu: support new bootloaders shipped by Marvell

I have already reviewed it internally. All the changes are really limited to mvebu and do not impact any other subsystem. So you have my acked-by, and once Jason Cooper will have given his Acked-by, then I think we will be able to apply the series on mvebu/ branches.

Thanks,

Quoted 14 lines
 arch/arm/boot/dts/armada-370-xp.dtsi |    2 +-
 arch/arm/boot/dts/armada-370.dtsi    |    3 +-
 arch/arm/boot/dts/armada-xp-gp.dts   |    2 +-
 arch/arm/include/debug/mvebu.S       |   77 +++++++++++++++++++--
 arch/arm/mach-mvebu/armada-370-xp.c  |  124 +++++++++++++++++++++++++++++-----
 arch/arm/mach-mvebu/armada-370-xp.h  |    4 +-
 arch/arm/mach-mvebu/coherency.c      |   36 ++++------
 arch/arm/mach-mvebu/coherency.h      |    4 --
 arch/arm/mach-mvebu/common.h         |    2 +
 arch/arm/mach-mvebu/headsmp.S        |   20 +++---
 arch/arm/mach-mvebu/platsmp.c        |   10 ++-
 11 files changed, 220 insertions(+), 64 deletions(-)
Signature · 4 lines
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com