Clean out SPL and enable driver model

This function has grown into something of a monster. Some boards are setting up a console and DRAM here in SPL. This requires global_data which should be set up in one place (crt0.S).

There is no need for SPL to use s_init() for anything since board_init_f() is called immediately afterwards.

Signed-off-by Simon Glass <[email protected]>

This is already set up in crt0.S. We don't need a new structure and don't really want one in the 'data' section of the image, since it will be empty and crt0.S's changes will be ignored.

As an interim measure, remove it only if CONFIG_DM is not defined. This allows us to press ahead with driver model in SPL and allow the stragglers to catch up.

Signed-off-by Simon Glass <[email protected]>

Use the full driver model GPIO and serial drivers in SPL now that these are supported. Since device tree is not available they will use platform data.

Remove the special SPL GPIO function as it is no longer needed.

This is all in one commit to maintain bisectability.

Signed-off-by Simon Glass <[email protected]>

Quoted 4 lines · Simon Glass

Use the full driver model GPIO and serial drivers in SPL now that these are supported. Since device tree is not available they will use platform data.

Quoted 5 lines

Remove the special SPL GPIO function as it is no longer needed.

This is all in one commit to maintain bisectability.

Signed-off-by Simon Glass <[email protected]>

Applied to u-boot/master, thanks!

Quoted 10 lines · Tom Rini

Quoted 8 lines · Simon Glass

Use the full driver model GPIO and serial drivers in SPL now that these are supported. Since device tree is not available they will use platform data.

Remove the special SPL GPIO function as it is no longer needed.

This is all in one commit to maintain bisectability.

Quoted 3 lines

Quoted 1 line

Signed-off-by Simon Glass <[email protected]>

Applied to u-boot/master, thanks!

This patch breaks the serial console on at least NVIDIA Tegra Seaboard. LCD/usbkbd still work though. I can't test any other Tegra boards right now since I'm not at work. I wonder if the UART/SPI GPIO is at issue? That would be specific to this board.

(I just happened to test with that board while investigating why usbkbd was not working right on the RPi, since Seaboard has usbkbd support... Yay nested bugs!)

Hi Stephen,

Quoted 16 lines · Stephen Warren

Quoted 13 lines · Tom Rini

Quoted 9 lines · Simon Glass

Use the full driver model GPIO and serial drivers in SPL now that these are supported. Since device tree is not available they will use platform data.

Remove the special SPL GPIO function as it is no longer needed.

This is all in one commit to maintain bisectability.

Signed-off-by Simon Glass <[email protected]>

Applied to u-boot/master, thanks!

Quoted 4 lines

This patch breaks the serial console on at least NVIDIA Tegra Seaboard. LCD/usbkbd still work though. I can't test any other Tegra boards right now since I'm not at work. I wonder if the UART/SPI GPIO is at issue? That would be specific to this board.

Could be - what do you have it set to? I should be able to give it another test sometime today and nut it out.

Quoted 24 lines · Simon Glass

Hi Stephen,

Quoted 20 lines · Stephen Warren

Quoted 15 lines · Tom Rini

Quoted 9 lines · Simon Glass

Use the full driver model GPIO and serial drivers in SPL now that these are supported. Since device tree is not available they will use platform data.

Remove the special SPL GPIO function as it is no longer needed.

This is all in one commit to maintain bisectability.

Signed-off-by Simon Glass <[email protected]>

Applied to u-boot/master, thanks!

Quoted 8 lines

Quoted 4 lines

This patch breaks the serial console on at least NVIDIA Tegra Seaboard. LCD/usbkbd still work though. I can't test any other Tegra boards right now since I'm not at work. I wonder if the UART/SPI GPIO is at issue? That would be specific to this board.

Could be - what do you have it set to? I should be able to give it another test sometime today and nut it out.

I assume it must be set to "GPIO control" (especially since a GPIO-related change broke it). IIRC it came this way, and since the board boots from NAND and hence SPI isn't useful, I was going to change it to "always UART". However, when I attempted to modify my other Seaboard (actually, Springbank) that way, I broke one of the pads on the PCB. Luckily that left the board in the "always UART" state that I wanted, even without any pull-ups/downs present. Anyway, I decided not to touch my other board for fear of breaking it in a way that didn't work at all!

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is documented in the README.

Signed-off-by Simon Glass <[email protected]>

For version 1:

Acked-by Albert ARIBAUD <[email protected]>

Reviewed-by Stefan Roese <[email protected]>

Tested-by Bo Shen <[email protected]>

Acked-by Bo Shen <[email protected]>

Acked-by Heiko Schocher <[email protected]>

Tested-by Heiko Schocher <[email protected]>

---

Changes in v6: None
Changes in v5:
- Rebase to master

Changes in v4:
- Adjust README to mention that lowlevel_init() should have no stack

Changes in v3: None
Changes in v2:
- Move docs to top-level README file and expand them to cover U-Boot proper
- Add Kconfig settings

 Kconfig             | 18 ++++++++++++++
 README              | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 arch/arm/lib/crt0.S | 13 +++++++---
 common/spl/spl.c    | 35 +++++++++++++++++++++++++++
 4 files changed, 132 insertions(+), 3 deletions(-)

arch/arm/lib/crt0.S +10 -3

index 22df3e5..7939ced 100644
@@ -113,7 +113,14 @@ here:
 /* Set up final (full) environment */
 
 	bl	c_runtime_cpu_setup	/* we still call old routine here */
-
+#endif
+#if !defined(CONFIG_SPL_BUILD) || defined(CONFIG_SPL_FRAMEWORK)
+# ifdef CONFIG_SPL_BUILD
+	/* Use a DRAM stack for the rest of SPL, if requested */
+	bl	spl_relocate_stack_gd
+	cmp	r0, #0
+	movne	sp, r0
+# endif
 	ldr	r0, =__bss_start	/* this is auto-relocated! */
 	ldr	r1, =__bss_end		/* this is auto-relocated! */
 

@@ -124,9 +131,10 @@ clbss_l:cmp	r0, r1			/* while not at end of BSS */
 	addlo	r0, r0, #4		/* move to next */
 	blo	clbss_l
 
+#if ! defined(CONFIG_SPL_BUILD)
 	bl coloured_LED_init
 	bl red_led_on
-
+#endif
 	/* call board_init_r(gd_t *id, ulong dest_addr) */
 	mov     r0, r9                  /* gd_t */
 	ldr	r1, [r9, #GD_RELOCADDR]	/* dest_addr */

@@ -134,7 +142,6 @@ clbss_l:cmp	r0, r1			/* while not at end of BSS */
 	ldr	pc, =board_init_r	/* this is auto-relocated! */
 
 	/* we should not return here. */
-
 #endif
 
 ENDPROC(_main)

common/spl/spl.c +25 -0

index ded0f30..cd75bbc 100644
@@ -281,3 +281,38 @@ void preloader_console_init(void)
 	spl_display_print();
 #endif
 }
+
+/**
+ * spl_relocate_stack_gd() - Relocate stack ready for board_init_r() execution
+ *
+ * Sometimes board_init_f() runs with a stack in SRAM but we want to use SDRAM
+ * for the main board_init_r() execution. This is typically because we need
+ * more stack space for things like the MMC sub-system.
+ *
+ * This function calculates the stack position, copies the global_data into
+ * place and returns the new stack position. The caller is responsible for
+ * setting up the sp register.
+ *
+ * @return new stack location, or 0 to use the same stack
+ */
+ulong spl_relocate_stack_gd(void)
+{
+#ifdef CONFIG_SPL_STACK_R
+	gd_t *new_gd;
+	ulong ptr;
+
+	/* Get stack position: use 8-byte alignment for ABI compliance */
+	ptr = CONFIG_SPL_STACK_R - sizeof(gd_t);
+	ptr &= ~7;
+	new_gd = (gd_t *)ptr;
+	memcpy(new_gd, (void *)gd, sizeof(gd_t));

Quoted 44 lines · Simon Glass

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is documented in the README.

Signed-off-by Simon Glass <[email protected]>

For version 1:

Acked-by Albert ARIBAUD <[email protected]>

Reviewed-by Stefan Roese <[email protected]>

Tested-by Bo Shen <[email protected]>

Acked-by Bo Shen <[email protected]>

Acked-by Heiko Schocher <[email protected]>

Tested-by Heiko Schocher <[email protected]>

---

Changes in v6: None
Changes in v5:
- Rebase to master

Changes in v4:
- Adjust README to mention that lowlevel_init() should have no stack

Changes in v3: None
Changes in v2:
- Move docs to top-level README file and expand them to cover U-Boot proper
- Add Kconfig settings

 Kconfig             | 18 ++++++++++++++
 README              | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 arch/arm/lib/crt0.S | 13 +++++++---
 common/spl/spl.c    | 35 +++++++++++++++++++++++++++
 4 files changed, 132 insertions(+), 3 deletions(-)

[snip]

Quoted 81 lines

arch/arm/lib/crt0.S +10 -3

index 22df3e5..7939ced 100644
@@ -113,7 +113,14 @@ here:
 /* Set up final (full) environment */
 
      bl      c_runtime_cpu_setup     /* we still call old routine here */
-
+#endif
+#if !defined(CONFIG_SPL_BUILD) || defined(CONFIG_SPL_FRAMEWORK)
+# ifdef CONFIG_SPL_BUILD
+     /* Use a DRAM stack for the rest of SPL, if requested */
+     bl      spl_relocate_stack_gd
+     cmp     r0, #0
+     movne   sp, r0
+# endif
      ldr     r0, =__bss_start        /* this is auto-relocated! */
      ldr     r1, =__bss_end          /* this is auto-relocated! */
 

@@ -124,9 +131,10 @@ clbss_l:cmp      r0, r1                  /* while not at end of BSS */
      addlo   r0, r0, #4              /* move to next */
      blo     clbss_l
 
+#if ! defined(CONFIG_SPL_BUILD)
      bl coloured_LED_init
      bl red_led_on
-
+#endif
      /* call board_init_r(gd_t *id, ulong dest_addr) */
      mov     r0, r9                  /* gd_t */
      ldr     r1, [r9, #GD_RELOCADDR] /* dest_addr */

@@ -134,7 +142,6 @@ clbss_l:cmp       r0, r1                  /* while not at end of BSS */
      ldr     pc, =board_init_r       /* this is auto-relocated! */
 
      /* we should not return here. */
-
 #endif
 
 ENDPROC(_main)

common/spl/spl.c +33 -0

index ded0f30..cd75bbc 100644
@@ -281,3 +281,38 @@ void preloader_console_init(void)
      spl_display_print();
 #endif
 }
+
+/**
+ * spl_relocate_stack_gd() - Relocate stack ready for board_init_r() execution
+ *
+ * Sometimes board_init_f() runs with a stack in SRAM but we want to use SDRAM
+ * for the main board_init_r() execution. This is typically because we need
+ * more stack space for things like the MMC sub-system.
+ *
+ * This function calculates the stack position, copies the global_data into
+ * place and returns the new stack position. The caller is responsible for
+ * setting up the sp register.
+ *
+ * @return new stack location, or 0 to use the same stack
+ */
+ulong spl_relocate_stack_gd(void)
+{
+#ifdef CONFIG_SPL_STACK_R
+     gd_t *new_gd;
+     ulong ptr;
+
+     /* Get stack position: use 8-byte alignment for ABI compliance */
+     ptr = CONFIG_SPL_STACK_R - sizeof(gd_t);
+     ptr &= ~7;
+     new_gd = (gd_t *)ptr;
+     memcpy(new_gd, (void *)gd, sizeof(gd_t));
+     gd = new_gd;
+
+     /* Clear the BSS. */
+     memset(__bss_start, 0, __bss_end - __bss_start);
+
+     return ptr;
+#else
+     return 0;

common/spl/spl.c void preloader_console_init(void) · line 318

Quoted 2 lines

common/spl/spl.c +2 -0

+#endif
+}

All of this _does_ move gd into where CONFIG_SPL_STACK_R points. It does _not_ move the stack itself into where CONFIG_SPL_STACK_R points so the big use case (am335x_boneblack for example where CONFIG_SPL_ENV_SUPPORT is set) doesn't work and blows up as we overflow the area for stack.

Hi Tom,

Quoted 128 lines · Tom Rini

Quoted 44 lines · Simon Glass

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is documented in the README.

Signed-off-by Simon Glass <[email protected]>

For version 1:

Acked-by Albert ARIBAUD <[email protected]>

Reviewed-by Stefan Roese <[email protected]>

Tested-by Bo Shen <[email protected]>

Acked-by Bo Shen <[email protected]>

Acked-by Heiko Schocher <[email protected]>

Tested-by Heiko Schocher <[email protected]>

---

Changes in v6: None
Changes in v5:
- Rebase to master

Changes in v4:
- Adjust README to mention that lowlevel_init() should have no stack

Changes in v3: None
Changes in v2:
- Move docs to top-level README file and expand them to cover U-Boot proper
- Add Kconfig settings

 Kconfig             | 18 ++++++++++++++
 README              | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 arch/arm/lib/crt0.S | 13 +++++++---
 common/spl/spl.c    | 35 +++++++++++++++++++++++++++
 4 files changed, 132 insertions(+), 3 deletions(-)

[snip]

Quoted 81 lines

arch/arm/lib/crt0.S +10 -3

index 22df3e5..7939ced 100644
@@ -113,7 +113,14 @@ here:
 /* Set up final (full) environment */
 
      bl      c_runtime_cpu_setup     /* we still call old routine here */
-
+#endif
+#if !defined(CONFIG_SPL_BUILD) || defined(CONFIG_SPL_FRAMEWORK)
+# ifdef CONFIG_SPL_BUILD
+     /* Use a DRAM stack for the rest of SPL, if requested */
+     bl      spl_relocate_stack_gd
+     cmp     r0, #0
+     movne   sp, r0
+# endif
      ldr     r0, =__bss_start        /* this is auto-relocated! */
      ldr     r1, =__bss_end          /* this is auto-relocated! */
 

@@ -124,9 +131,10 @@ clbss_l:cmp      r0, r1                  /* while not at end of BSS */
      addlo   r0, r0, #4              /* move to next */
      blo     clbss_l
 
+#if ! defined(CONFIG_SPL_BUILD)
      bl coloured_LED_init
      bl red_led_on
-
+#endif
      /* call board_init_r(gd_t *id, ulong dest_addr) */
      mov     r0, r9                  /* gd_t */
      ldr     r1, [r9, #GD_RELOCADDR] /* dest_addr */

@@ -134,7 +142,6 @@ clbss_l:cmp       r0, r1                  /* while not at end of BSS */
      ldr     pc, =board_init_r       /* this is auto-relocated! */
 
      /* we should not return here. */
-
 #endif
 
 ENDPROC(_main)

common/spl/spl.c +33 -0

index ded0f30..cd75bbc 100644
@@ -281,3 +281,38 @@ void preloader_console_init(void)
      spl_display_print();
 #endif
 }
+
+/**
+ * spl_relocate_stack_gd() - Relocate stack ready for board_init_r() execution
+ *
+ * Sometimes board_init_f() runs with a stack in SRAM but we want to use SDRAM
+ * for the main board_init_r() execution. This is typically because we need
+ * more stack space for things like the MMC sub-system.
+ *
+ * This function calculates the stack position, copies the global_data into
+ * place and returns the new stack position. The caller is responsible for
+ * setting up the sp register.
+ *
+ * @return new stack location, or 0 to use the same stack
+ */
+ulong spl_relocate_stack_gd(void)
+{
+#ifdef CONFIG_SPL_STACK_R
+     gd_t *new_gd;
+     ulong ptr;
+
+     /* Get stack position: use 8-byte alignment for ABI compliance */
+     ptr = CONFIG_SPL_STACK_R - sizeof(gd_t);
+     ptr &= ~7;
+     new_gd = (gd_t *)ptr;
+     memcpy(new_gd, (void *)gd, sizeof(gd_t));
+     gd = new_gd;
+
+     /* Clear the BSS. */
+     memset(__bss_start, 0, __bss_end - __bss_start);
+
+     return ptr;
+#else
+     return 0;

Quoted 8 lines

common/spl/spl.c void preloader_console_init(void) · line 318

Quoted 2 lines

common/spl/spl.c +2 -0

+#endif
+}

All of this _does_ move gd into where CONFIG_SPL_STACK_R points. It does _not_ move the stack itself into where CONFIG_SPL_STACK_R points so the big use case (am335x_boneblack for example where CONFIG_SPL_ENV_SUPPORT is set) doesn't work and blows up as we overflow the area for stack.

OK I'll have to test more. What sort of problem should i see?

The return value from spl_relocate_stack_gd() should be shoved into sp. Does that not happen?

Quoted 140 lines · Simon Glass

Hi Tom,

Quoted 128 lines · Tom Rini

Quoted 44 lines · Simon Glass

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is documented in the README.

Signed-off-by Simon Glass <[email protected]>

For version 1:

Acked-by Albert ARIBAUD <[email protected]>

Reviewed-by Stefan Roese <[email protected]>

Tested-by Bo Shen <[email protected]>

Acked-by Bo Shen <[email protected]>

Acked-by Heiko Schocher <[email protected]>

Tested-by Heiko Schocher <[email protected]>

---

Changes in v6: None
Changes in v5:
- Rebase to master

Changes in v4:
- Adjust README to mention that lowlevel_init() should have no stack

Changes in v3: None
Changes in v2:
- Move docs to top-level README file and expand them to cover U-Boot proper
- Add Kconfig settings

 Kconfig             | 18 ++++++++++++++
 README              | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 arch/arm/lib/crt0.S | 13 +++++++---
 common/spl/spl.c    | 35 +++++++++++++++++++++++++++
 4 files changed, 132 insertions(+), 3 deletions(-)

[snip]

Quoted 81 lines

arch/arm/lib/crt0.S +10 -3

index 22df3e5..7939ced 100644
@@ -113,7 +113,14 @@ here:
 /* Set up final (full) environment */
 
      bl      c_runtime_cpu_setup     /* we still call old routine here */
-
+#endif
+#if !defined(CONFIG_SPL_BUILD) || defined(CONFIG_SPL_FRAMEWORK)
+# ifdef CONFIG_SPL_BUILD
+     /* Use a DRAM stack for the rest of SPL, if requested */
+     bl      spl_relocate_stack_gd
+     cmp     r0, #0
+     movne   sp, r0
+# endif
      ldr     r0, =__bss_start        /* this is auto-relocated! */
      ldr     r1, =__bss_end          /* this is auto-relocated! */
 

@@ -124,9 +131,10 @@ clbss_l:cmp      r0, r1                  /* while not at end of BSS */
      addlo   r0, r0, #4              /* move to next */
      blo     clbss_l
 
+#if ! defined(CONFIG_SPL_BUILD)
      bl coloured_LED_init
      bl red_led_on
-
+#endif
      /* call board_init_r(gd_t *id, ulong dest_addr) */
      mov     r0, r9                  /* gd_t */
      ldr     r1, [r9, #GD_RELOCADDR] /* dest_addr */

@@ -134,7 +142,6 @@ clbss_l:cmp       r0, r1                  /* while not at end of BSS */
      ldr     pc, =board_init_r       /* this is auto-relocated! */
 
      /* we should not return here. */
-
 #endif
 
 ENDPROC(_main)

common/spl/spl.c +33 -0

index ded0f30..cd75bbc 100644
@@ -281,3 +281,38 @@ void preloader_console_init(void)
      spl_display_print();
 #endif
 }
+
+/**
+ * spl_relocate_stack_gd() - Relocate stack ready for board_init_r() execution
+ *
+ * Sometimes board_init_f() runs with a stack in SRAM but we want to use SDRAM
+ * for the main board_init_r() execution. This is typically because we need
+ * more stack space for things like the MMC sub-system.
+ *
+ * This function calculates the stack position, copies the global_data into
+ * place and returns the new stack position. The caller is responsible for
+ * setting up the sp register.
+ *
+ * @return new stack location, or 0 to use the same stack
+ */
+ulong spl_relocate_stack_gd(void)
+{
+#ifdef CONFIG_SPL_STACK_R
+     gd_t *new_gd;
+     ulong ptr;
+
+     /* Get stack position: use 8-byte alignment for ABI compliance */
+     ptr = CONFIG_SPL_STACK_R - sizeof(gd_t);
+     ptr &= ~7;
+     new_gd = (gd_t *)ptr;
+     memcpy(new_gd, (void *)gd, sizeof(gd_t));
+     gd = new_gd;
+
+     /* Clear the BSS. */
+     memset(__bss_start, 0, __bss_end - __bss_start);
+
+     return ptr;
+#else
+     return 0;

Quoted 8 lines

common/spl/spl.c void preloader_console_init(void) · line 318

Quoted 2 lines

common/spl/spl.c +2 -0

+#endif
+}

All of this _does_ move gd into where CONFIG_SPL_STACK_R points. It does _not_ move the stack itself into where CONFIG_SPL_STACK_R points so the big use case (am335x_boneblack for example where CONFIG_SPL_ENV_SUPPORT is set) doesn't work and blows up as we overflow the area for stack.

Quoted 1 line

OK I'll have to test more. What sort of problem should i see?

We hang on entering env_reloc_spec() which is what puts some big sized variables on the stack.

Quoted 2 lines

The return value from spl_relocate_stack_gd() should be shoved into sp. Does that not happen?

It doesn't appear to be. I hadn't broken out the BBB with JTAG tho.

Hi Tom,

Quoted 149 lines · Tom Rini

Quoted 140 lines · Simon Glass

Hi Tom,

Quoted 128 lines · Tom Rini

Quoted 44 lines · Simon Glass

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is documented in the README.

Signed-off-by Simon Glass <[email protected]>

For version 1:

Acked-by Albert ARIBAUD <[email protected]>

Reviewed-by Stefan Roese <[email protected]>

Tested-by Bo Shen <[email protected]>

Acked-by Bo Shen <[email protected]>

Acked-by Heiko Schocher <[email protected]>

Tested-by Heiko Schocher <[email protected]>

---

Changes in v6: None
Changes in v5:
- Rebase to master

Changes in v4:
- Adjust README to mention that lowlevel_init() should have no stack

Changes in v3: None
Changes in v2:
- Move docs to top-level README file and expand them to cover U-Boot proper
- Add Kconfig settings

 Kconfig             | 18 ++++++++++++++
 README              | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 arch/arm/lib/crt0.S | 13 +++++++---
 common/spl/spl.c    | 35 +++++++++++++++++++++++++++
 4 files changed, 132 insertions(+), 3 deletions(-)

[snip]

Quoted 81 lines

arch/arm/lib/crt0.S +10 -3

index 22df3e5..7939ced 100644
@@ -113,7 +113,14 @@ here:
 /* Set up final (full) environment */
 
      bl      c_runtime_cpu_setup     /* we still call old routine here */
-
+#endif
+#if !defined(CONFIG_SPL_BUILD) || defined(CONFIG_SPL_FRAMEWORK)
+# ifdef CONFIG_SPL_BUILD
+     /* Use a DRAM stack for the rest of SPL, if requested */
+     bl      spl_relocate_stack_gd
+     cmp     r0, #0
+     movne   sp, r0
+# endif
      ldr     r0, =__bss_start        /* this is auto-relocated! */
      ldr     r1, =__bss_end          /* this is auto-relocated! */
 

@@ -124,9 +131,10 @@ clbss_l:cmp      r0, r1                  /* while not at end of BSS */
      addlo   r0, r0, #4              /* move to next */
      blo     clbss_l
 
+#if ! defined(CONFIG_SPL_BUILD)
      bl coloured_LED_init
      bl red_led_on
-
+#endif
      /* call board_init_r(gd_t *id, ulong dest_addr) */
      mov     r0, r9                  /* gd_t */
      ldr     r1, [r9, #GD_RELOCADDR] /* dest_addr */

@@ -134,7 +142,6 @@ clbss_l:cmp       r0, r1                  /* while not at end of BSS */
      ldr     pc, =board_init_r       /* this is auto-relocated! */
 
      /* we should not return here. */
-
 #endif
 
 ENDPROC(_main)

common/spl/spl.c +33 -0

index ded0f30..cd75bbc 100644
@@ -281,3 +281,38 @@ void preloader_console_init(void)
      spl_display_print();
 #endif
 }
+
+/**
+ * spl_relocate_stack_gd() - Relocate stack ready for board_init_r() execution
+ *
+ * Sometimes board_init_f() runs with a stack in SRAM but we want to use SDRAM
+ * for the main board_init_r() execution. This is typically because we need
+ * more stack space for things like the MMC sub-system.
+ *
+ * This function calculates the stack position, copies the global_data into
+ * place and returns the new stack position. The caller is responsible for
+ * setting up the sp register.
+ *
+ * @return new stack location, or 0 to use the same stack
+ */
+ulong spl_relocate_stack_gd(void)
+{
+#ifdef CONFIG_SPL_STACK_R
+     gd_t *new_gd;
+     ulong ptr;
+
+     /* Get stack position: use 8-byte alignment for ABI compliance */
+     ptr = CONFIG_SPL_STACK_R - sizeof(gd_t);
+     ptr &= ~7;
+     new_gd = (gd_t *)ptr;
+     memcpy(new_gd, (void *)gd, sizeof(gd_t));
+     gd = new_gd;
+
+     /* Clear the BSS. */
+     memset(__bss_start, 0, __bss_end - __bss_start);
+
+     return ptr;
+#else
+     return 0;

Quoted 8 lines

common/spl/spl.c void preloader_console_init(void) · line 318

Quoted 2 lines

common/spl/spl.c +2 -0

+#endif
+}

All of this _does_ move gd into where CONFIG_SPL_STACK_R points. It does _not_ move the stack itself into where CONFIG_SPL_STACK_R points so the big use case (am335x_boneblack for example where CONFIG_SPL_ENV_SUPPORT is set) doesn't work and blows up as we overflow the area for stack.

Quoted 1 line

OK I'll have to test more. What sort of problem should i see?

We hang on entering env_reloc_spec() which is what puts some big sized variables on the stack.

Quoted 2 lines

The return value from spl_relocate_stack_gd() should be shoved into sp. Does that not happen?

Quoted 1 line

It doesn't appear to be. I hadn't broken out the BBB with JTAG tho.

I printed out the stack in board_init_r() and it looks right: 32MB above the base of SDRAM.

Also I don't seem to get a crash in SPL. How do I make that happen? If I define CONFIG_SPL_USBETH_SUPPORT I get an error about the .data section not fitin in .sram for u-boot-spl.

Quoted 157 lines · Simon Glass

Hi Tom,

Quoted 143 lines · Tom Rini

Quoted 140 lines · Simon Glass

Hi Tom,

Quoted 128 lines · Tom Rini

Quoted 44 lines · Simon Glass

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is documented in the README.

Signed-off-by Simon Glass <[email protected]>

For version 1:

Acked-by Albert ARIBAUD <[email protected]>

Reviewed-by Stefan Roese <[email protected]>

Tested-by Bo Shen <[email protected]>

Acked-by Bo Shen <[email protected]>

Acked-by Heiko Schocher <[email protected]>

Tested-by Heiko Schocher <[email protected]>

---

Changes in v6: None
Changes in v5:
- Rebase to master

Changes in v4:
- Adjust README to mention that lowlevel_init() should have no stack

Changes in v3: None
Changes in v2:
- Move docs to top-level README file and expand them to cover U-Boot proper
- Add Kconfig settings

 Kconfig             | 18 ++++++++++++++
 README              | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 arch/arm/lib/crt0.S | 13 +++++++---
 common/spl/spl.c    | 35 +++++++++++++++++++++++++++
 4 files changed, 132 insertions(+), 3 deletions(-)

[snip]

Quoted 81 lines

arch/arm/lib/crt0.S +10 -3

index 22df3e5..7939ced 100644
@@ -113,7 +113,14 @@ here:
 /* Set up final (full) environment */
 
      bl      c_runtime_cpu_setup     /* we still call old routine here */
-
+#endif
+#if !defined(CONFIG_SPL_BUILD) || defined(CONFIG_SPL_FRAMEWORK)
+# ifdef CONFIG_SPL_BUILD
+     /* Use a DRAM stack for the rest of SPL, if requested */
+     bl      spl_relocate_stack_gd
+     cmp     r0, #0
+     movne   sp, r0
+# endif
      ldr     r0, =__bss_start        /* this is auto-relocated! */
      ldr     r1, =__bss_end          /* this is auto-relocated! */
 

@@ -124,9 +131,10 @@ clbss_l:cmp      r0, r1                  /* while not at end of BSS */
      addlo   r0, r0, #4              /* move to next */
      blo     clbss_l
 
+#if ! defined(CONFIG_SPL_BUILD)
      bl coloured_LED_init
      bl red_led_on
-
+#endif
      /* call board_init_r(gd_t *id, ulong dest_addr) */
      mov     r0, r9                  /* gd_t */
      ldr     r1, [r9, #GD_RELOCADDR] /* dest_addr */

@@ -134,7 +142,6 @@ clbss_l:cmp       r0, r1                  /* while not at end of BSS */
      ldr     pc, =board_init_r       /* this is auto-relocated! */
 
      /* we should not return here. */
-
 #endif
 
 ENDPROC(_main)

common/spl/spl.c +33 -0

index ded0f30..cd75bbc 100644
@@ -281,3 +281,38 @@ void preloader_console_init(void)
      spl_display_print();
 #endif
 }
+
+/**
+ * spl_relocate_stack_gd() - Relocate stack ready for board_init_r() execution
+ *
+ * Sometimes board_init_f() runs with a stack in SRAM but we want to use SDRAM
+ * for the main board_init_r() execution. This is typically because we need
+ * more stack space for things like the MMC sub-system.
+ *
+ * This function calculates the stack position, copies the global_data into
+ * place and returns the new stack position. The caller is responsible for
+ * setting up the sp register.
+ *
+ * @return new stack location, or 0 to use the same stack
+ */
+ulong spl_relocate_stack_gd(void)
+{
+#ifdef CONFIG_SPL_STACK_R
+     gd_t *new_gd;
+     ulong ptr;
+
+     /* Get stack position: use 8-byte alignment for ABI compliance */
+     ptr = CONFIG_SPL_STACK_R - sizeof(gd_t);
+     ptr &= ~7;
+     new_gd = (gd_t *)ptr;
+     memcpy(new_gd, (void *)gd, sizeof(gd_t));
+     gd = new_gd;
+
+     /* Clear the BSS. */
+     memset(__bss_start, 0, __bss_end - __bss_start);
+
+     return ptr;
+#else
+     return 0;

Quoted 8 lines

common/spl/spl.c void preloader_console_init(void) · line 318

Quoted 2 lines

common/spl/spl.c +2 -0

+#endif
+}

All of this _does_ move gd into where CONFIG_SPL_STACK_R points. It does _not_ move the stack itself into where CONFIG_SPL_STACK_R points so the big use case (am335x_boneblack for example where CONFIG_SPL_ENV_SUPPORT is set) doesn't work and blows up as we overflow the area for stack.

Quoted 1 line

OK I'll have to test more. What sort of problem should i see?

Quoted 7 lines

We hang on entering env_reloc_spec() which is what puts some big sized variables on the stack.

Quoted 2 lines

The return value from spl_relocate_stack_gd() should be shoved into sp. Does that not happen?

It doesn't appear to be. I hadn't broken out the BBB with JTAG tho.

I printed out the stack in board_init_r() and it looks right: 32MB above the base of SDRAM.

Quoted 3 lines

Also I don't seem to get a crash in SPL. How do I make that happen? If I define CONFIG_SPL_USBETH_SUPPORT I get an error about the .data section not fitin in .sram for u-boot-spl.

With v5 at least, am335x_boneblack_config running on a BBB does it. I'll try v6 this morning and see what's going on..

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is documented in the README.

Kconfig +3 -0

index 91a0618..8de731d 100644
@@ -96,6 +96,24 @@ config SPL
 	help
 	  If you want to build SPL as well as the normal image, say Y.
 
+config CONFIG_SPL_STACK_R
+	depends on SPL
+	bool "Enable SDRAM location for SPL stack"

Quoted 16 lines · Simon Glass

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is documented in the README.

[snip]

Quoted 11 lines

Kconfig +3 -0

index 91a0618..8de731d 100644
@@ -96,6 +96,24 @@ config SPL
      help
        If you want to build SPL as well as the normal image, say Y.
 
+config CONFIG_SPL_STACK_R
+     depends on SPL
+     bool "Enable SDRAM location for SPL stack"

Kconfig config SPL · line 111

Quoted 10 lines

Kconfig +10 -0

+     help
+       SPL starts off execution in SRAM and thus typically has only a small
+       stack available. Since SPL sets up DRAM while in its board_init_f()
+       function, it is possible for the stack to move there before
+       board_init_r() is reached. This option enables a special SDRAM
+       location for the SPL stack. U-Boot SPL switches to this after
+       board_init_f() completes, and before board_init_r() starts.
+
+config CONFIG_SPL_STACK_R_ADDR
+     depends on CONFIG_SPL_STACK_R

I found my problem! Incorrect Kconfig syntax, no CONFIG_ in 'config' or 'depends' lines only help lines. So I was right in that my stack wasn't moving because I was doing it via the defconfig and that was broken :)

I'm just going to pick up 1-5 (with this fixed) and send out my own 6 after I test it on omap3 and omap4/5 which also need a similar conversion.

At present SPL uses a single stack, either CONFIG_SPL_STACK or CONFIG_SYS_INIT_SP_ADDR. Since some SPL features (such as MMC and environment) require a lot of stack, some boards set CONFIG_SPL_STACK to point into SDRAM. They then set up SDRAM very early, before board_init_f(), so that the larger stack can be used.

This is an abuse of lowlevel_init(). That function should only be used for essential start-up code which cannot be delayed. An example of a valid use is when only part of the SPL code is visible/executable, and the SoC must be set up so that board_init_f() can be reached. It should not be used for SDRAM init, console init, etc.

Add a CONFIG_SPL_STACK_R option, which allows the stack to be moved to a new address before board_init_r() is called in SPL.

The expected SPL flow (for CONFIG_SPL_FRAMEWORK) is documented in the README.

Signed-off-by Simon Glass <[email protected]>

For version 1:

Acked-by Albert ARIBAUD <[email protected]>

Reviewed-by Stefan Roese <[email protected]>

Tested-by Bo Shen <[email protected]>

Acked-by Bo Shen <[email protected]>

Acked-by Heiko Schocher <[email protected]>

Tested-by Heiko Schocher <[email protected]>

When CONFIG_SYS_MALLOC_SIMPLE is defined, free() is a static inline. Make sure that the export interface still builds in this case.

Signed-off-by Simon Glass <[email protected]>

Currently SDRAM is set up before global_data is available. This will soon not work. Adjust the ordering to cope, using the new CONFIG_SPL_STACK_R option to ensure that the stack is still in SDRAM once baord_init_r() starts in SPL.

Signed-off-by Simon Glass <[email protected]>

Quoted 5 lines · Simon Glass

Currently SDRAM is set up before global_data is available. This will soon not work. Adjust the ordering to cope, using the new CONFIG_SPL_STACK_R option to ensure that the stack is still in SDRAM once baord_init_r() starts in SPL.

Quoted 1 line

Signed-off-by Simon Glass <[email protected]>

It's good that we're on the same path :) I think we should stick this into the config files as part of getting people used to that. What I've got locally will also remove a bunch of CONFIG_SPL_STACK settings that don't do anything now (since they pointed back up to CONFIG_SYS_INIT_SP_ADDR). I'm going to test the hard case (falcon mode poking at env for settings).

But I'm going to grab your patch as the base since keeping board_early_init_f() around will keep the am335x+NOR case closer to working.

From: Simon Glass <[email protected]>

Currently in some cases SDRAM init requires global_data to be available and soon this will not be available prior to board_init_f(). Adjust the code paths in these cases to be correct. In some cases we had the SPL stack be in DDR as we might have large stacks (due to Falcon Mode + Environment). In these cases switch to CONFIG_SPL_STACK_R. In other cases we had simply been setting CONFIG_SPL_STACK into SRAM. In these cases we no longer need to (CONFIG_SYS_INIT_SP_ADDR is used and is also in SRAM) so drop those lines.

Signed-off-by Simon Glass <[email protected]>

Tested on Beagleboard, Beagleboard xM

Tested-by Matt Porter <[email protected]>

Tested on Beaglebone Black, AM43xx GP EVM, OMAP5 uEVM, OMAP4 Pandaboard

Tested-by Tom Rini <[email protected]>

Signed-off-by Tom Rini <[email protected]>

From: Simon Glass <[email protected]>

Currently in some cases SDRAM init requires global_data to be available and soon this will not be available prior to board_init_f(). Adjust the code paths in these cases to be correct. In some cases we had the SPL stack be in DDR as we might have large stacks (due to Falcon Mode + Environment). In these cases switch to CONFIG_SPL_STACK_R. In other cases we had simply been setting CONFIG_SPL_STACK into SRAM. In these cases we no longer need to (CONFIG_SYS_INIT_SP_ADDR is used and is also in SRAM) so drop those lines.

Signed-off-by Simon Glass <[email protected]>

Tested on Beagleboard, Beagleboard xM

Tested-by Matt Porter <[email protected]>

Tested on Beaglebone Black, AM43xx GP EVM, OMAP5 uEVM, OMAP4 Pandaboard

Tested-by Tom Rini <[email protected]>

Signed-off-by Tom Rini <[email protected]>