arm: spl: Allow board_init_r() to run with a larger stack

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is now:

Execution starts with start.S. Two main functions can be provided by the board implementation. The purpose and limitations of each is described below. After that, the common board_init_r() is called to perform the SPL task.

lowlevel_init():
- purpose: essential init to permit execution to reach board_init_f()
- no global_data, but there is a stack

- must not set up SDRAM or use console
- must only do the bare minimum to allow execution to continue to board_init_f()
- this is almost never needed
- return normally from this function

board_init_f():
- purpose: set up the machine ready for running board_init_r():
i.e. SDRAM and serial UART
- global_data is available
- preloader_console_init() can be called here in extremis
- stack is in SRAM
- should set up SDRAM, and anything needed to make the UART work
- these is no need to clear BSS, it will be done by crt0.S
- must return normally from this function (don't call board_init_r() directly)

Here the BSS is cleared. If CONFIG_SPL_STACK_R is defined, then at this point the stack and global_data are relocated to below that address.

board_init_r():
- purpose: main execution, common code
- global_data is available
- SDRAM is available
- stack is optionally in SDRAM, if CONFIG_SPL_STACK_R is defined and points into SDRAM
- preloader_console_init() can be called here - typically this is done by defining CONFIG_SPL_BOARD_INIT and then supplying a spl_board_init() function containing this call
- loads U-Boot or (in falcon mode) Linux

Signed-off-by Simon Glass <[email protected]>

For version 1:

Acked-by Albert ARIBAUD <[email protected]>

Reviewed-by Stefan Roese <[email protected]>

Tested-by Bo Shen <[email protected]>

Acked-by Bo Shen <[email protected]>

Acked-by Heiko Schocher <[email protected]>

Tested-by Heiko Schocher <[email protected]>

---

Changes in v2:
- Move docs to top-level README file and expand them to cover U-Boot proper
- Add Kconfig settings

 Kconfig             | 18 ++++++++++++++
 README              | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 arch/arm/lib/crt0.S | 13 +++++++---
 common/spl/spl.c    | 35 +++++++++++++++++++++++++++
 4 files changed, 131 insertions(+), 3 deletions(-)

Kconfig +18 -0

index 4157da3..8398bcd 100644
@@ -92,6 +92,24 @@ config SPL
 	help
 	  If you want to build SPL as well as the normal image, say Y.
 
+config CONFIG_SPL_STACK_R
+	depends on SPL
+	bool "Enable SDRAM location for SPL stack"
+	help
+	  SPL starts off execution in SRAM and thus typically has only a small
+	  stack available. Since SPL sets up DRAM while in its board_init_f()
+	  function, it is possible for the stack to move there before
+	  board_init_r() is reached. This option enables a special SDRAM
+	  location for the SPL stack. U-Boot SPL switches to this after
+	  board_init_f() completes, and before board_init_r() starts.
+
+config CONFIG_SPL_STACK_R_ADDR
+	depends on CONFIG_SPL_STACK_R
+	hex "SDRAM location for SPL stack"
+	help
+	  Specify the address in SDRAM for the SPL stack. This will be set up
+	  before board_init_r() is called.
+
 config TPL
 	bool
 	depends on SPL && SUPPORT_TPL

README +68 -0

index fefa71c..7b48cda 100644
@@ -273,6 +273,74 @@ run some of U-Boot's tests.
 See board/sandbox/README.sandbox for more details.
 
 
+Board Initialisation Flow:
+--------------------------
+
+This is the intended start-up flow for boards. This should apply for both
+SPL and U-Boot proper (i.e. they both follow the same rules). At present SPL
+mostly uses a separate code path, but the funtion names and roles of each
+function are the same. Some boards or architectures may not conform to this.
+At least most ARM boards which use CONFIG_SPL_FRAMEWORK conform to this.
+
+Execution starts with start.S with three functions called during init after
+that. The purpose and limitations of each is described below.
+
+lowlevel_init():
+	- purpose: essential init to permit execution to reach board_init_f()
+	- no global_data or BSS, but there is a stack
+	- must not set up SDRAM or use console
+	- must only do the bare minimum to allow execution to continue to
+		board_init_f()
+	- this is almost never needed
+	- return normally from this function
+
+board_init_f():
+	- purpose: set up the machine ready for running board_init_r():
+		i.e. SDRAM and serial UART
+	- global_data is available
+	- stack is in SRAM
+	- BSS is not available, so you cannot use global/static variables,
+		only stack variables and global_data
+
+	Non-SPL-specific notes:
+	- dram_init() is called to set up DRAM. If already done in SPL this
+		can do nothing
+
+	SPL-specific notes:
+	- you can override the entire board_init_f() function with your own
+		version as needed.
+	- preloader_console_init() can be called here in extremis
+	- should set up SDRAM, and anything needed to make the UART work
+	- these is no need to clear BSS, it will be done by crt0.S
+	- must return normally from this function (don't call board_init_r()
+		directly)
+
+Here the BSS is cleared. For SPL, if CONFIG_SPL_STACK_R is defined, then at
+this point the stack and global_data are relocated to below
+CONFIG_SPL_STACK_R_ADDR. For non-SPL, U-Boot is relocated to run at the top of
+memory.
+
+board_init_r():
+	- purpose: main execution, common code
+	- global_data is available
+	- SDRAM is available
+	- BSS is available, all static/global variables can be used
+	- execution eventually continues to main_loop()
+
+	Non-SPL-specific notes:
+	- U-Boot is relocated to the top of memory and is now running from
+		there.
+
+	SPL-specific notes:
+	- stack is optionally in SDRAM, if CONFIG_SPL_STACK_R is defined and
+		CONFIG_SPL_STACK_R_ADDR points into SDRAM
+	- preloader_console_init() can be called here - typically this is
+		done by defining CONFIG_SPL_BOARD_INIT and then supplying a
+		spl_board_init() function containing this call
+	- loads U-Boot or (in falcon mode) Linux
+
+
+
 Configuration Options:
 ----------------------
 

arch/arm/lib/crt0.S +10 -3

index 22df3e5..7939ced 100644
@@ -113,7 +113,14 @@ here:
 /* Set up final (full) environment */
 
 	bl	c_runtime_cpu_setup	/* we still call old routine here */
-
+#endif
+#if !defined(CONFIG_SPL_BUILD) || defined(CONFIG_SPL_FRAMEWORK)
+# ifdef CONFIG_SPL_BUILD
+	/* Use a DRAM stack for the rest of SPL, if requested */
+	bl	spl_relocate_stack_gd
+	cmp	r0, #0
+	movne	sp, r0
+# endif
 	ldr	r0, =__bss_start	/* this is auto-relocated! */
 	ldr	r1, =__bss_end		/* this is auto-relocated! */
 

@@ -124,9 +131,10 @@ clbss_l:cmp	r0, r1			/* while not at end of BSS */
 	addlo	r0, r0, #4		/* move to next */
 	blo	clbss_l
 
+#if ! defined(CONFIG_SPL_BUILD)
 	bl coloured_LED_init
 	bl red_led_on
-
+#endif
 	/* call board_init_r(gd_t *id, ulong dest_addr) */
 	mov     r0, r9                  /* gd_t */
 	ldr	r1, [r9, #GD_RELOCADDR]	/* dest_addr */

@@ -134,7 +142,6 @@ clbss_l:cmp	r0, r1			/* while not at end of BSS */
 	ldr	pc, =board_init_r	/* this is auto-relocated! */
 
 	/* we should not return here. */
-
 #endif
 
 ENDPROC(_main)

common/spl/spl.c +35 -0

index 1826c47..78bb279 100644
@@ -276,3 +276,38 @@ void preloader_console_init(void)
 	spl_display_print();
 #endif
 }
+
+/**
+ * spl_relocate_stack_gd() - Relocate stack ready for board_init_r() execution
+ *
+ * Sometimes board_init_f() runs with a stack in SRAM but we want to use SDRAM
+ * for the main board_init_r() execution. This is typically because we need
+ * more stack space for things like the MMC sub-system.
+ *
+ * This function calculates the stack position, copies the global_data into
+ * place and returns the new stack position. The caller is responsible for
+ * setting up the sp register.
+ *
+ * @return new stack location, or 0 to use the same stack
+ */
+ulong spl_relocate_stack_gd(void)
+{
+#ifdef CONFIG_SPL_STACK_R
+	gd_t *new_gd;
+	ulong ptr;
+
+	/* Get stack position: use 8-byte alignment for ABI compliance */
+	ptr = CONFIG_SPL_STACK_R - sizeof(gd_t);
+	ptr &= ~7;
+	new_gd = (gd_t *)ptr;
+	memcpy(new_gd, (void *)gd, sizeof(gd_t));
+	gd = new_gd;
+
+	/* Clear the BSS. */
+	memset(__bss_start, 0, __bss_end - __bss_start);
+
+	return ptr;
+#else
+	return 0;
+#endif
+}

Signature · 1 line

2.2.0.rc0.207.ga3a616c

Hello Simon,

Quoted 17 lines · Simon Glass

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is now:

Quoted 7 lines

Execution starts with start.S. Two main functions can be provided by the board implementation. The purpose and limitations of each is described below. After that, the common board_init_r() is called to perform the SPL task.

lowlevel_init():
- purpose: essential init to permit execution to reach board_init_f()
- no global_data, but there is a stack

lowlevel_init() does not have a stack for all ARM CPUs, e.g. for arm926ejs it has none; and I don't see any code in this patch that provides one. If we need lowlevel_init to have a stack, then this stack should be set up in crt0.S and lowlevel_init called from there (which I would be fine with, btw).

Hi Albert,

Quoted 34 lines · Albert ARIBAUD

Hello Simon,

Quoted 17 lines · Simon Glass

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is now:

Quoted 7 lines

Execution starts with start.S. Two main functions can be provided by the board implementation. The purpose and limitations of each is described below. After that, the common board_init_r() is called to perform the SPL task.

lowlevel_init():
- purpose: essential init to permit execution to reach board_init_f()
- no global_data, but there is a stack

lowlevel_init() does not have a stack for all ARM CPUs, e.g. for arm926ejs it has none; and I don't see any code in this patch that provides one. If we need lowlevel_init to have a stack, then this stack should be set up in crt0.S and lowlevel_init called from there (which I would be fine with, btw).

OK, so I should adjust this comment I think. Is that the only action?

- no global_data, but may be a stack (e.g. for armv7)

Quoted 37 lines · Simon Glass

Hi Albert,

Quoted 34 lines · Albert ARIBAUD

Hello Simon,

Quoted 17 lines · Simon Glass

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is now:

Quoted 7 lines

Execution starts with start.S. Two main functions can be provided by the board implementation. The purpose and limitations of each is described below. After that, the common board_init_r() is called to perform the SPL task.

lowlevel_init():
- purpose: essential init to permit execution to reach board_init_f()
- no global_data, but there is a stack

lowlevel_init() does not have a stack for all ARM CPUs, e.g. for arm926ejs it has none; and I don't see any code in this patch that provides one. If we need lowlevel_init to have a stack, then this stack should be set up in crt0.S and lowlevel_init called from there (which I would be fine with, btw).

Keep in mind that today in crt0.S we basically setup stack and call board_init_f(). I'm not sure, but maybe it would be, useful to have a pre-board_init_f hook? But maybe it's more globally useful rather than an ARM thing.

Quoted 5 lines

OK, so I should adjust this comment I think. Is that the only action?

- no global_data, but may be a stack (e.g. for armv7)

Looking at all of this code recently, for trying to consolidate things, I think we need to figure out the right answer for how to deal with things like perhaps FEL too and be clear about what we do and don't want. The pre-armv7 arches seem rather consistent but armv7 went off on its own thing for a while and is slowly cycling back now.

I'm not sure, but what to do with FEL will help guide us, that we might be able to say lowlevel_init does not have stack, even on armv7. If we can shove a hook early into board_init_r() and can delay those things that we do early today.

Hi Tom,

Quoted 39 lines · Tom Rini

Quoted 37 lines · Simon Glass

Hi Albert,

Quoted 34 lines · Albert ARIBAUD

Hello Simon,

Quoted 17 lines · Simon Glass

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is now:

Quoted 7 lines

Execution starts with start.S. Two main functions can be provided by the board implementation. The purpose and limitations of each is described below. After that, the common board_init_r() is called to perform the SPL task.

lowlevel_init():
- purpose: essential init to permit execution to reach board_init_f()
- no global_data, but there is a stack

lowlevel_init() does not have a stack for all ARM CPUs, e.g. for arm926ejs it has none; and I don't see any code in this patch that provides one. If we need lowlevel_init to have a stack, then this stack should be set up in crt0.S and lowlevel_init called from there (which I would be fine with, btw).

Quoted 4 lines

Keep in mind that today in crt0.S we basically setup stack and call board_init_f(). I'm not sure, but maybe it would be, useful to have a pre-board_init_f hook? But maybe it's more globally useful rather than an ARM thing.

I feel it would be nice to do this from crt0.S rather than start.S, since it is common code. Any time we have funny things in individual start.S files it creates tension.

Quoted 10 lines

Quoted 3 lines

OK, so I should adjust this comment I think. Is that the only action?

- no global_data, but may be a stack (e.g. for armv7)

Looking at all of this code recently, for trying to consolidate things, I think we need to figure out the right answer for how to deal with things like perhaps FEL too and be clear about what we do and don't want. The pre-armv7 arches seem rather consistent but armv7 went off on its own thing for a while and is slowly cycling back now.

Quoted 4 lines

I'm not sure, but what to do with FEL will help guide us, that we might be able to say lowlevel_init does not have stack, even on armv7. If we can shove a hook early into board_init_r() and can delay those things that we do early today.

Declaring that lowlevel_init() may not have a stack would be one way to stop people adding more and more junk there :-)

I am going to take a close look at FEL. I feel that FEL is a good test of the flexibility of the early phase: start.S, lowlevel_init / board_init_f() - particularly for SPL.