Add support for dynamic MMU configuration
Subsystems (5)
v11 cover Add support for dynamic MMU configuration
Hi all,
In U-Boot, TI only provides a single memory map for all k3 platforms, this does not scale for devices where atf and optee lie outside the range 0x80000000
- 0x80080000 and 0x9e780000 - 0xa0000000 respectively.
There are also issues for devices with < 2GiB of memory (eg am62SiP with 512MiB of RAM) as the maximum size for the first DRAM bank is hardcoded to 2GiB in the current memory map. Furthermore the second DRAM bank is mapped even for devices that only have a single bank.
Therefore this patch set adds the required functionality to create the MMU table at runtime based on the device-tree.
The patch set has been build tested on all effected platforms but boot-tested only on TI's K3 EVMs, the beagleplay and phytec's phycore-am6* platforms.
The following effected boards have not been boot tested:
- verdin-am62
- iot2050
Best Wishes,
Anshul
---
Changes in v11:
- Rename spl_perform_fixups for am62dx
- Add 'check_nomap' to mmu_unmap_reserved_mem
v10: https://lore.kernel.org/u-boot/[email protected]/
Changes in v10:
- Rename mem_map_fix_dram_banks to mem_map_from_dram_banks
- Use log_err instead of printf
- Add check for "no-map" in the reserved-memory node before unmapping it
v9: https://lore.kernel.org/u-boot/[email protected]/
Changes in v9:
- Pick R-by tags
- Change error messages from debug to printf
v8: https://lore.kernel.org/u-boot/[email protected]/
Changes in v8:
- Reserve extra space for page table entries for performing fixups
v7: https://lore.kernel.org/u-boot/[email protected]/
Changes in v7:
- Pick up R-by and Tested-by tags
- Use nobreak API for unmapping MMU region
- Better error messages and minor refactor
v6: https://lore.kernel.org/u-boot/[email protected]/
Changes in v6:
- Make use of generic MMU APIs
- Extend core U-Boot functionality by adding mmu_unmap_reserved_mem and mem_map_fix_dram_banks
- Keep same memory maps for SPL and U-Boot proper
- Add fdt fixups from SPL stage for reserved memory nodes
v5: https://lore.kernel.org/u-boot/[email protected]/
Changes in v5:
- Don't create carveouts for every reserved-memory node
- Only create carveouts for ATF/OP-TEE
- Expand the call to k3_mem_map_init to vendor boards as well
- Map area for framebuffer for CONFIG_VIDEO=y platforms
v4: https://lore.kernel.org/u-boot/[email protected]/
Changes for v4:
- Add call to k3_mem_map_init for beagleplay
- Mark reserved regions as non-cacheable
- More debug logs
v3: https://lore.kernel.org/u-boot/[email protected]/
Changes for v3:
- Remove unused memory regions in SPL's map
- Add runtime addition of MMU entry for the framebuffer in SPL
- Refactor k3_mem_map_init to use standard u-boot APIs
- Unmap reserved-memory regions instead of keeping them uncached
v2: https://lore.kernel.org/u-boot/[email protected]/
Changes in v2:
- Removed dependency to:
https://lore.kernel.org/u-boot/[email protected]/
v1: https://lore.kernel.org/u-boot/[email protected]/
---
Anshul Dalal (11):
mach-k3: use minimal memory map for all K3
mach-k3: use custom enable_cache
arm: armv8: mmu: export mmu_setup
arm: armv8: invalidate dcache entries on dcache_enable
arm: armv8: mmu: add mem_map_from_dram_banks
mach-k3: map all banks using mem_map_from_dram_banks
arm: armv8: mmu: add mmu_unmap_reserved_mem
spl: split spl_board_fixups to arch/board specific
mach-k3: add reserved memory fixups for next boot stage
mach-k3: add carveouts for TFA and optee
arm: mach-k3: reserve space for page table entries
arch/arm/cpu/armv7m/cpu.c | 2 +- arch/arm/cpu/armv8/cache_v8.c | 61 ++++++++++++++++++++-- arch/arm/include/asm/armv8/mmu.h | 25 +++++++++ arch/arm/mach-k3/arm64/arm64-mmu.c | 40 ++++++-------- arch/arm/mach-k3/common.c | 55 +++++++++++++++++++ arch/arm/mach-k3/include/mach/k3-ddr.h | 6 +++ arch/arm/mach-rockchip/spl-boot-order.c | 2 +- arch/arm/mach-socfpga/spl_soc64.c | 2 +- board/beagle/beagley-ai/beagley-ai.c | 2 +- board/dhelectronics/dh_stm32mp1/board.c | 2 +- board/phytec/phycore_am62x/phycore-am62x.c | 2 +- board/phytec/phycore_am64x/phycore-am64x.c | 2 +- board/renesas/sparrowhawk/sparrowhawk.c | 2 +- board/starfive/visionfive2/spl.c | 2 +- board/ti/am62ax/evm.c | 2 +- board/ti/am62dx/evm.c | 2 +- board/ti/am62px/evm.c | 2 +- board/ti/am62x/evm.c | 2 +- board/ti/am64x/evm.c | 2 +- board/ti/j721e/evm.c | 2 +- board/ti/j721s2/evm.c | 2 +- board/ti/j722s/evm.c | 2 +- board/ti/j784s4/evm.c | 2 +- common/spl/spl.c | 12 +++-- include/spl.h | 12 +++-- 25 files changed, 196 insertions(+), 51 deletions(-)
Signature · 1 line
2.51.0
PATCH v11 1/11 mach-k3: use minimal memory map for all K3
The K3 family of SoCs encompasses a wide variety of devices with varying DDR configurations and memory carveout requirements, the current static memory map provides basic support for TI EVMs but does not scale well for newer platforms (such as AM62SiP with 512MiB of RAM).
Therefore this patch replaces the existing memory map with a minimal one, that could be more easily modified at runtime.
--- arch/arm/mach-k3/arm64/arm64-mmu.c | 32 ++++++++++-------------------- 1 file changed, 10 insertions(+), 22 deletions(-)
arch/arm/mach-k3/arm64/arm64-mmu.c +10 -22
@@ -11,42 +11,30 @@#include <asm/system.h> #include <asm/armv8/mmu.h> +#include <linux/sizes.h> struct mm_region k3_mem_map[] = { - { + {/* SoC Peripherals */ .virt =0x0UL , .phys =0x0UL , .size =0x80000000UL , .attrs =PTE_BLOCK_MEMTYPE (MT_DEVICE_NGNRNE ) |PTE_BLOCK_NON_SHARE |PTE_BLOCK_PXN |PTE_BLOCK_UXN - }, { - .virt =0x80000000UL , - .phys =0x80000000UL , - .size =0x1e780000UL , - .attrs =PTE_BLOCK_MEMTYPE (MT_NORMAL ) | -PTE_BLOCK_INNER_SHARE - }, { - .virt =0xa0000000UL , - .phys =0xa0000000UL , - .size =0x60000000UL , - .attrs =PTE_BLOCK_MEMTYPE (MT_NORMAL ) | -PTE_BLOCK_INNER_SHARE - }, { - .virt =0x880000000UL , - .phys =0x880000000UL , - .size =0x80000000UL , - .attrs =PTE_BLOCK_MEMTYPE (MT_NORMAL ) | -PTE_BLOCK_INNER_SHARE - }, { + }, {/* Flash Peripherals */ .virt =0x500000000UL , .phys =0x500000000UL , .size =0x380000000UL , .attrs =PTE_BLOCK_MEMTYPE (MT_DEVICE_NGNRNE ) |PTE_BLOCK_NON_SHARE |PTE_BLOCK_PXN |PTE_BLOCK_UXN - }, { -/* List terminator */ + }, {/* First DRAM Bank of size 2G */ + .virt =CFG_SYS_SDRAM_BASE , + .phys =CFG_SYS_SDRAM_BASE , + .size =SZ_2G , + .attrs =PTE_BLOCK_MEMTYPE (MT_NORMAL ) | +PTE_BLOCK_INNER_SHARE + }, {/* List terminator */ 0 , } };
Signature · 1 line
2.51.0
PATCH v11 2/11 mach-k3: use custom enable_cache
In reply to Anshul DalalU-Boot's provided enable_caches enforces the following sequence:icache_enable -> mmu_setup (as part of dcache_enable) -> dcache_enable
Whereas for K3 devices, we would like to add entries to the provided static array (k3_mem_map) as per gd->bd->bi_dram and then call mmu_setup to configure the MMU but also create carveouts for TFA/TEE before we enable caches to prevent speculative accesses to the region. Thus the following desired sequence:
add dram banks -> mmu_setup -> carveout TFA/TEE -> icache/dcache enable
Therefore this patch adds K3's own implementation of enable_cache for ARM64, allowing for greater control over the cache enablement sequence.
--- arch/arm/mach-k3/common.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/arm/mach-k3/common.c +6 -0
@@ -258,6 +258,12 @@ void board_prep_linux(struct bootm_headers *images)ROUND (images->os.end, CONFIG_SYS_CACHELINE_SIZE)); } + +void enable_caches (void ) +{ +icache_enable (); +dcache_enable (); +}#endif void spl_enable_cache (void )
Signature · 1 line
2.51.0
PATCH v11 3/11 arm: armv8: mmu: export mmu_setup
In reply to Anshul DalalThe mmu_setup function configures the page tables based on the board supplied mem_map struct array. It is called implicitly as part of dcache_enable but this limits us to only be able to use APIs such as mmu_change_region_attr only after caches are enabled.
This might lead to speculative accesses before we can unmap a region that is marked as cacheable in the static memory map.
Therefore this patch exports the mmu_setup function in mmu.h allowing users to have more control over when the mmu is configured.
For K3 specifically this allows for the following configuration sequence as part of enable_caches:
static mem_map fixups (TODO) -> mmu_setup -> carveouts using mmu_change_region_attr (TODO) -> icache/dcache enable
--- arch/arm/include/asm/armv8/mmu.h | 5 +++++ arch/arm/mach-k3/common.c | 3 +++ 2 files changed, 8 insertions(+)
arch/arm/include/asm/armv8/mmu.h +5 -0
@@ -197,6 +197,11 @@ struct mm_region {extern struct mm_region *mem_map;void setup_pgtables (void );u64 get_tcr (u64 *pips,u64 *pva_bits); + +/** +* mmu_setup() - Sets up the mmu page tables as per mem_map +*/ +void mmu_setup (void );#endif #endif /* _ASM_ARMV8_MMU_H_ */
arch/arm/mach-k3/common.c +3 -0
@@ -30,6 +30,7 @@#include <soc.h> #include <dm/uclass-internal.h> #include <dm/device-internal.h> +#include <asm/armv8/mmu.h> #define PROC_BOOT_CTRL_FLAG_R5_CORE_HALT 0x00000001#define PROC_BOOT_STATUS_FLAG_R5_WFI 0x00000002
@@ -261,6 +262,8 @@ void board_prep_linux(struct bootm_headers *images)void enable_caches (void ) { +mmu_setup (); +icache_enable ();dcache_enable (); }
Signature · 1 line
2.51.0
PATCH v11 4/11 arm: armv8: invalidate dcache entries on dcache_enable
In reply to Anshul DalalIn dcache_enable, currently the dcache entries are only invalidated when the MMU is not enabled. This causes issues when dcache_enable is called with the MMU already configured, in such cases the existing dcache entries are not flushed which might result in un-expected behavior.
This patch invalidates the cache entries on every call of dcache_enable before enabling dcache (by setting CR_C). This makes dcache_enable behave similar to icache_enable as well.
--- arch/arm/cpu/armv8/cache_v8.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
arch/arm/cpu/armv8/cache_v8.c +3 -4
@@ -830,16 +830,15 @@ void flush_dcache_range(unsigned long start, unsigned long stop)void dcache_enable (void ) {/* The data cache is not active unless the mmu is enabled */ -if (!(get_sctlr () &CR_M )) { -invalidate_dcache_all (); -__asm_invalidate_tlb_all (); +if (!mmu_status ())mmu_setup (); - }/* Set up page tables only once (it is done also by mmu_setup()) */ if (!gd->arch.tlb_fillptr)setup_all_pgtables (); +invalidate_dcache_all (); +__asm_invalidate_tlb_all ();set_sctlr (get_sctlr () |CR_C ); }
Signature · 1 line
2.51.0
PATCH v11 5/11 arm: armv8: mmu: add mem_map_from_dram_banks
In reply to Anshul DalalFor armv8, U-Boot uses a static map defined as 'mem_map' for configuring the MMU as part of mmu_setup.
But since the exact configuration of memory banks might not be known at build time, many platforms such as imx9, versal2 etc. utilize gd->bd->bi_dram to configure the static map at runtime.
Therefore this patch adds a new API mem_map_from_dram_banks that modifies the static map in a similar way. Allowing the caller to map all dram banks by just passing the index to last entry in their mem_map and it's length.
--- arch/arm/cpu/armv8/cache_v8.c | 28 ++++++++++++++++++++++++++++ arch/arm/include/asm/armv8/mmu.h | 11 +++++++++++ 2 files changed, 39 insertions(+)
arch/arm/cpu/armv8/cache_v8.c +28 -0
@@ -58,6 +58,34 @@ static int get_effective_el(void)return el; } +int mem_map_from_dram_banks (unsigned int index,unsigned int len,u64 attrs) +{ +unsigned int i; +int ret =fdtdec_setup_memory_banksize (); + +if (ret) { +log_err ("%s: Failed to setup dram banks\n" , __func__); +return ret; + } + +if (index +CONFIG_NR_DRAM_BANKS >= len) { +log_err ("%s: Provided mem_map array has insufficient size for DRAM entries\n" , + __func__); +return -ENOMEM ; + } + +for (i =0 ; i <CONFIG_NR_DRAM_BANKS ; i++) { + mem_map[index].virt = gd->bd->bi_dram[i].start; + mem_map[index].phys = gd->bd->bi_dram[i].start; + mem_map[index].size = gd->bd->bi_dram[i].size; + mem_map[index].attrs = attrs; + index++; + } + +memset (&mem_map[index],0 ,sizeof (mem_map[index])); + +return 0 ; +}u64 get_tcr (u64 *pips,u64 *pva_bits) {int el =get_effective_el ();
arch/arm/include/asm/armv8/mmu.h +11 -0
@@ -194,8 +194,19 @@ struct mm_region {u64 attrs; }; +/* Used as the memory map for MMU configuration by mmu_setup */ extern struct mm_region *mem_map;void setup_pgtables (void ); + +/** +* mem_map_from_dram_banks() - Populate mem_map with entries corresponding to +* dram banks as per the gd. This should be called prior to mmu_setup. +* +* @index: The entry in mem_map to start the over-write +* @len: The size of mem_map +*/ +int mem_map_from_dram_banks (unsigned int index,unsigned int len,u64 attrs); +u64 get_tcr (u64 *pips,u64 *pva_bits);/**
Signature · 1 line
2.51.0
PATCH v11 6/11 mach-k3: map all banks using mem_map_from_dram_banks
In reply to Anshul DalalThe static memory map for K3 (k3_mem_map) only maps the first DRAM bank and therefore doesn't scale for platforms with multiple memory banks.
This patch modifies enable_caches to add mem_map_from_dram_banks which appends all the memory banks to k3_mem_map before calling mmu_setup.
--- arch/arm/mach-k3/arm64/arm64-mmu.c | 5 +++-- arch/arm/mach-k3/common.c | 9 +++++++++ arch/arm/mach-k3/include/mach/k3-ddr.h | 6 ++++++ 3 files changed, 18 insertions(+), 2 deletions(-)
arch/arm/mach-k3/arm64/arm64-mmu.c +3 -2
@@ -12,8 +12,9 @@#include <asm/system.h> #include <asm/armv8/mmu.h> #include <linux/sizes.h> +#include <mach/k3-ddr.h> -struct mm_region k3_mem_map[] = { +struct mm_region k3_mem_map[K3_MEM_MAP_LEN ] = { {/* SoC Peripherals */ .virt =0x0UL , .phys =0x0UL ,
@@ -28,7 +29,7 @@ struct mm_region k3_mem_map[] = { .attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) | PTE_BLOCK_NON_SHARE | PTE_BLOCK_PXN | PTE_BLOCK_UXN - }, { /* First DRAM Bank of size 2G */ + }, [K3_MEM_MAP_FIRST_BANK_IDX] = { /* First DRAM Bank of size 2G */ .virt = CFG_SYS_SDRAM_BASE, .phys = CFG_SYS_SDRAM_BASE, .size = SZ_2G,
arch/arm/mach-k3/common.c +9 -0
@@ -31,6 +31,7 @@#include <dm/uclass-internal.h> #include <dm/device-internal.h> #include <asm/armv8/mmu.h> +#include <mach/k3-ddr.h> #define PROC_BOOT_CTRL_FLAG_R5_CORE_HALT 0x00000001#define PROC_BOOT_STATUS_FLAG_R5_WFI 0x00000002
@@ -262,6 +263,14 @@ void board_prep_linux(struct bootm_headers *images)void enable_caches (void ) { +int ret; + + ret =mem_map_from_dram_banks (K3_MEM_MAP_FIRST_BANK_IDX ,K3_MEM_MAP_LEN , +PTE_BLOCK_MEMTYPE (MT_NORMAL ) | +PTE_BLOCK_INNER_SHARE ); +if (ret) +debug ("%s: Failed to setup dram banks\n" , __func__); +mmu_setup ();icache_enable ();
arch/arm/mach-k3/include/mach/k3-ddr.h +6 -0
@@ -8,6 +8,12 @@#include <spl.h> +/* We need 3 extra entries for: +* SoC peripherals, flash and the sentinel value. +*/ +#define K3_MEM_MAP_LEN ((CONFIG_NR_DRAM_BANKS) + 3) +#define K3_MEM_MAP_FIRST_BANK_IDX 2 +int dram_init (void );int dram_init_banksize (void );
Signature · 1 line
2.51.0
PATCH v11 7/11 arm: armv8: mmu: add mmu_unmap_reserved_mem
In reply to Anshul DalalFor armv8, U-Boot uses a static map defined as 'mem_map' for configuring the MMU's page tables, done by mmu_setup.
Though this works well for simpler platforms, it makes creating runtime carveouts by modifying the static array at runtime exceedingly complex like in mach-snapdragon/board.c.
Creation of such carveouts are much better handled by APIs such as mmu_change_region_attr once the page tables are configured. Usually such carveouts are configured via the device-tree's reserved-memory node which provides the address and size for the carveout.
Therefore this patch adds mmu_unmap_reserved_mem which acts as a wrapper over mmu_change_region_attr, helping unmap a reserved-memory region.
--- arch/arm/cpu/armv8/cache_v8.c | 26 ++++++++++++++++++++++++++ arch/arm/include/asm/armv8/mmu.h | 9 +++++++++ 2 files changed, 35 insertions(+)
arch/arm/cpu/armv8/cache_v8.c +26 -0
@@ -86,6 +86,32 @@ int mem_map_from_dram_banks(unsigned int index, unsigned int len, u64 attrs)return 0 ; } + +int mmu_unmap_reserved_mem (const char *name,bool check_nomap) +{ +void *fdt = (void *)gd->fdt_blob; +char node_path[128 ]; +fdt_addr_t addr; +fdt_size_t size; +int ret; + +snprintf (node_path,sizeof (node_path),"/reserved-memory/%s" , name); + ret =fdt_path_offset (fdt, node_path); +if (ret <0 ) +return ret; + +if (check_nomap && !fdtdec_get_bool (fdt, ret,"no-map" )) +return -EINVAL ; + + addr =fdtdec_get_addr_size (fdt, ret,"reg" , &size); +if (addr ==FDT_ADDR_T_NONE ) +return -1 ; + +mmu_change_region_attr_nobreak (addr, size,PTE_TYPE_FAULT ); + +return 0 ; +} +u64 get_tcr (u64 *pips,u64 *pva_bits) {int el =get_effective_el ();
arch/arm/include/asm/armv8/mmu.h +9 -0
@@ -207,6 +207,15 @@ void setup_pgtables(void);*/ int mem_map_from_dram_banks (unsigned int index,unsigned int len,u64 attrs); +/** +* mmu_unmap_reserved_mem() - Unmaps a reserved-memory node as PTE_TYPE_FAULT +* once MMU is configured by mmu_setup. +* +* @name: The name of the node under "/reserved-memory/" path +* @check_nomap: Check if the node is marked "no-map" before unmapping it +*/ +int mmu_unmap_reserved_mem (const char *name,bool check_nomap); +u64 get_tcr (u64 *pips,u64 *pva_bits);/**
Signature · 1 line
2.51.0
PATCH v11 8/11 spl: split spl_board_fixups to arch/board specific
In reply to Anshul DalalThe current spl_board_fixups API allows for modification of spl_image before the SPL jumps to it. This can be used to modify the DT for the next boot stage, however the current API only allows either the machine arch or the board to use it.
This limits the utility of the API as there might be certain fixups that should be applied to all boards sharing the same machine architecture with others being board specific.
For TI's K3 specifically, this prevents us from performing architecture level fixups since a lot of TI boards are already making use of the spl_board_fixups API.
Therefore this patch splits the API into two to allow both board and the architecture specific fixups. The order is kept as arch then board to give board specific fixups the precedence.
--- arch/arm/cpu/armv7m/cpu.c | 2 +- arch/arm/mach-rockchip/spl-boot-order.c | 2 +- arch/arm/mach-socfpga/spl_soc64.c | 2 +- board/beagle/beagley-ai/beagley-ai.c | 2 +- board/dhelectronics/dh_stm32mp1/board.c | 2 +- board/phytec/phycore_am62x/phycore-am62x.c | 2 +- board/phytec/phycore_am64x/phycore-am64x.c | 2 +- board/renesas/sparrowhawk/sparrowhawk.c | 2 +- board/starfive/visionfive2/spl.c | 2 +- board/ti/am62ax/evm.c | 2 +- board/ti/am62dx/evm.c | 2 +- board/ti/am62px/evm.c | 2 +- board/ti/am62x/evm.c | 2 +- board/ti/am64x/evm.c | 2 +- board/ti/j721e/evm.c | 2 +- board/ti/j721s2/evm.c | 2 +- board/ti/j722s/evm.c | 2 +- board/ti/j784s4/evm.c | 2 +- common/spl/spl.c | 12 +++++++++--- include/spl.h | 12 +++++++++--- 20 files changed, 36 insertions(+), 24 deletions(-)
arch/arm/cpu/armv7m/cpu.c +1 -1
@@ -57,7 +57,7 @@ void reset_cpu(void) | V7M_AIRCR_SYSRESET, &V7M_SCB->aircr); } -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_arch_fixups (struct spl_image_info *spl_image) { spl_image->entry_point |=0x1 ; }
arch/arm/mach-rockchip/spl-boot-order.c +1 -1
@@ -240,7 +240,7 @@ int spl_decode_boot_device(u32 boot_device, char *buf, size_t buflen)#endif } -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_arch_fixups (struct spl_image_info *spl_image) {const char *bootrom_ofpath =board_spl_was_booted_from ();void *blob =spl_image_fdt_addr (spl_image);
arch/arm/mach-socfpga/spl_soc64.c +1 -1
@@ -130,7 +130,7 @@ u32 spl_boot_mode(const u32 boot_device)#endif /* board specific function prior loading SSBL / U-Boot */ -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_arch_fixups (struct spl_image_info *spl_image) {int ret;struct udevice *dev;
board/beagle/beagley-ai/beagley-ai.c +1 -1
@@ -32,7 +32,7 @@ int dram_init_banksize(void) }#if defined(CONFIG_XPL_BUILD ) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {if (IS_ENABLED (CONFIG_K3_DDRSS )) {if (IS_ENABLED (CONFIG_K3_INLINE_ECC ))
board/dhelectronics/dh_stm32mp1/board.c +1 -1
@@ -776,7 +776,7 @@ int ft_board_setup(void *blob, struct bd_info *bd)#endif #if defined(CONFIG_XPL_BUILD ) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {dh_stm32_ks8851_fixup (spl_image_fdt_addr (spl_image)); }
board/phytec/phycore_am62x/phycore-am62x.c +1 -1
@@ -199,7 +199,7 @@ int do_board_detect(void)#endif #if IS_ENABLED (CONFIG_XPL_BUILD ) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {if (IS_ENABLED (CONFIG_K3_DDRSS ) &&IS_ENABLED (CONFIG_K3_INLINE_ECC ))fixup_ddr_driver_for_ecc (spl_image);
board/phytec/phycore_am64x/phycore-am64x.c +1 -1
@@ -118,7 +118,7 @@ int do_board_detect(void)#endif #if IS_ENABLED (CONFIG_XPL_BUILD ) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {if (IS_ENABLED (CONFIG_K3_DDRSS ) &&IS_ENABLED (CONFIG_K3_INLINE_ECC ))fixup_ddr_driver_for_ecc (spl_image);
board/renesas/sparrowhawk/sparrowhawk.c +1 -1
@@ -134,7 +134,7 @@ unsigned int spl_spi_get_uboot_offs(struct spi_flash *flash)return CONFIG_SYS_SPI_U_BOOT_OFFS ; } -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {void *blob =spl_image_fdt_addr (spl_image);int err, offs;
board/starfive/visionfive2/spl.c +1 -1
@@ -20,7 +20,7 @@ DECLARE_GLOBAL_DATA_PTR;#define JH7110_CLK_CPU_ROOT_SHIFT 24#define JH7110_CLK_CPU_ROOT_MASK GENMASK(29, 24) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {/* Update the memory size which read from eeprom or DT */ if (spl_image->fdt_addr)
board/ti/am62ax/evm.c +1 -1
@@ -17,7 +17,7 @@#include "../common/fdt_ops.h" #if defined(CONFIG_XPL_BUILD ) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {if (IS_ENABLED (CONFIG_K3_DDRSS )) {if (IS_ENABLED (CONFIG_K3_INLINE_ECC ))
board/ti/am62dx/evm.c +1 -1
@@ -17,7 +17,7 @@#include "../common/fdt_ops.h" #if defined(CONFIG_XPL_BUILD ) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {if (IS_ENABLED (CONFIG_K3_DDRSS )) {if (IS_ENABLED (CONFIG_K3_INLINE_ECC ))
board/ti/am62px/evm.c +1 -1
@@ -50,7 +50,7 @@ void spl_board_init(void)#endif #if defined(CONFIG_XPL_BUILD ) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {if (IS_ENABLED (CONFIG_K3_DDRSS )) {if (IS_ENABLED (CONFIG_K3_INLINE_ECC ))
board/ti/am62x/evm.c +1 -1
@@ -91,7 +91,7 @@ void spl_board_init(void) } -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {if (IS_ENABLED (CONFIG_K3_DDRSS )) {if (IS_ENABLED (CONFIG_K3_INLINE_ECC ))
board/ti/am64x/evm.c +1 -1
@@ -98,7 +98,7 @@ static int fixup_usb_boot(const void *fdt_blob) }#endif -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {if (IS_ENABLED (CONFIG_K3_DDRSS )) {if (IS_ENABLED (CONFIG_K3_INLINE_ECC ))
board/ti/j721e/evm.c +1 -1
@@ -127,7 +127,7 @@ static void __maybe_unused detect_enable_hyperflash(void *blob)#if defined(CONFIG_XPL_BUILD ) && (defined(CONFIG_TARGET_J7200_A72_EVM ) || defined(CONFIG_TARGET_J7200_R5_EVM ) || \ defined(CONFIG_TARGET_J721E_A72_EVM ) || defined(CONFIG_TARGET_J721E_R5_EVM )) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {detect_enable_hyperflash (spl_image->fdt_addr); }
board/ti/j721s2/evm.c +1 -1
@@ -40,7 +40,7 @@ phys_addr_t board_get_usable_ram_top(phys_size_t total_size) }#if defined(CONFIG_XPL_BUILD ) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {if (IS_ENABLED (CONFIG_K3_DDRSS )) {if (IS_ENABLED (CONFIG_K3_INLINE_ECC ))
board/ti/j722s/evm.c +1 -1
@@ -24,7 +24,7 @@ void spl_board_init(void)#endif #if defined(CONFIG_XPL_BUILD ) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {if (IS_ENABLED (CONFIG_K3_DDRSS )) {if (IS_ENABLED (CONFIG_K3_INLINE_ECC ))
board/ti/j784s4/evm.c +1 -1
@@ -42,7 +42,7 @@ struct efi_capsule_update_info update_info = { };#if defined(CONFIG_XPL_BUILD ) -void spl_perform_fixups (struct spl_image_info *spl_image) +void spl_perform_board_fixups (struct spl_image_info *spl_image) {if (IS_ENABLED (CONFIG_K3_DDRSS )) {if (IS_ENABLED (CONFIG_K3_INLINE_ECC ))
common/spl/spl.c +9 -3
@@ -120,8 +120,13 @@ int __weak booti_setup(ulong image, ulong *relocated_addr, ulong *size, bool for }#endif -/* Weak default function for arch/board-specific fixups to the spl_image_info */ -void __weak spl_perform_fixups (struct spl_image_info *spl_image) +/* Weak default function for arch specific fixups to the spl_image_info */ +void __weak spl_perform_arch_fixups (struct spl_image_info *spl_image) +{ +} + +/* Weak default function for board specific fixups to the spl_image_info */ +void __weak spl_perform_board_fixups (struct spl_image_info *spl_image) { }
@@ -775,7 +780,8 @@ void board_init_r(gd_t *dummy1, ulong dummy2)hang (); } -spl_perform_fixups (&spl_image); +spl_perform_arch_fixups (&spl_image); +spl_perform_board_fixups (&spl_image); os = spl_image.os;if (os ==IH_OS_U_BOOT ) {
include/spl.h +9 -3
@@ -1116,10 +1116,16 @@ int board_return_to_bootrom(struct spl_image_info *spl_image,ulong board_spl_fit_size_align (ulong size);/** -* spl_perform_fixups() - arch/board-specific callback before processing -* the boot-payload +* spl_perform_arch_fixups() - arch specific callback before processing the +* boot-payload */ -void spl_perform_fixups (struct spl_image_info *spl_image); +void spl_perform_arch_fixups (struct spl_image_info *spl_image); + +/** +* spl_perform_board_fixups() - board specific callback before processing the +* boot-payload +*/ +void spl_perform_board_fixups (struct spl_image_info *spl_image);/* * spl_get_load_buffer() - get buffer for loading partial image data
Signature · 1 line
2.51.0
PATCH v11 9/11 mach-k3: add reserved memory fixups for next boot stage
In reply to Anshul DalalThe boot flow in K3 platforms requires the subsequent boot stages to be aware of the memory reserved for previous boot binaries. In the regular boot sequence of:
R5 SPL -> ATF -> OP-TEE -> A-core SPL -> U-Boot -> Kernel,
Both A-core SPL and U-Boot should be made aware of the memory reserved for ATF and OP-TEE from their device-tree. Currently this information is absent.
Therefore this patch adds the reserved-memory regions as part of spl_perform_arch_fixups for the next stage's DT. This is called during both R5 and A-core SPL which fixes up the DT for their respective next
stages: A-core SPL and U-Boot proper.
--- arch/arm/mach-k3/common.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/arm/mach-k3/common.c +11 -0
@@ -321,6 +321,17 @@ static __maybe_unused void k3_dma_remove(void)pr_warn ("DMA Device not found (err=%d)\n" , rc); } +void spl_perform_arch_fixups (struct spl_image_info *spl_image) +{ +void *fdt =spl_image_fdt_addr (spl_image); + +if (!fdt) +return ; + +fdt_fixup_reserved (fdt,"tfa" ,CONFIG_K3_ATF_LOAD_ADDR ,0x80000 ); +fdt_fixup_reserved (fdt,"optee" ,CONFIG_K3_OPTEE_LOAD_ADDR ,0x1800000 ); +} +void spl_board_prepare_for_boot (void ) {#if !(defined(CONFIG_SYS_ICACHE_OFF ) && defined(CONFIG_SYS_DCACHE_OFF ))
Signature · 1 line
2.51.0
PATCH v11 10/11 mach-k3: add carveouts for TFA and optee
In reply to Anshul DalalK3 platforms have reserved memory regions for TFA and OPTEE which should be unmapped for U-Boot. While other "no-map" memory regions like the memory pools for remote cores should not be unmapped to allow U-Boot to load firmware during remoteproc.
Therefore this patch adds the necessary fdt fixups to properly set the load address for TFA/OPTEE and unmaps both by mmu_unmap_reserved_mem.
--- arch/arm/mach-k3/common.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
arch/arm/mach-k3/common.c +26 -0
@@ -31,6 +31,7 @@#include <dm/uclass-internal.h> #include <dm/device-internal.h> #include <asm/armv8/mmu.h> +#include <mach/k3-common-fdt.h> #include <mach/k3-ddr.h> #define PROC_BOOT_CTRL_FLAG_R5_CORE_HALT 0x00000001
@@ -263,6 +264,7 @@ void board_prep_linux(struct bootm_headers *images)void enable_caches (void ) { +void *fdt = (void *)gd->fdt_blob;int ret; ret = mem_map_from_dram_banks(K3_MEM_MAP_FIRST_BANK_IDX, K3_MEM_MAP_LEN,
@@ -273,6 +275,30 @@ void enable_caches(void)mmu_setup (); +if (CONFIG_K3_ATF_LOAD_ADDR >=CFG_SYS_SDRAM_BASE ) { + ret =fdt_fixup_reserved (fdt,"tfa" ,CONFIG_K3_ATF_LOAD_ADDR , +0x80000 ); +if (ret) +printf ("%s: Failed to perform tfa fixups (%s)\n" , + __func__,fdt_strerror (ret)); + ret =mmu_unmap_reserved_mem ("tfa" , true); +if (ret) +printf ("%s: Failed to unmap tfa reserved mem (%d)\n" , + __func__, ret); + } + +if (CONFIG_K3_OPTEE_LOAD_ADDR >=CFG_SYS_SDRAM_BASE ) { + ret =fdt_fixup_reserved (fdt,"optee" , +CONFIG_K3_OPTEE_LOAD_ADDR ,0x1800000 ); +if (ret) +printf ("%s: Failed to perform optee fixups (%s)\n" , + __func__,fdt_strerror (ret)); + ret =mmu_unmap_reserved_mem ("optee" , true); +if (ret) +printf ("%s: Failed to unmap optee reserved mem (%d)\n" , + __func__, ret); + } +icache_enable ();dcache_enable (); }
Signature · 1 line
2.51.0
PATCH v11 11/11 arm: mach-k3: reserve space for page table entries
In reply to Anshul DalalWith the memory map configuration being done dynamically, reserve extra space during U-Boot relocation to ensure we have enough for the fixups.
--- arch/arm/mach-k3/arm64/arm64-mmu.c | 5 +++++ 1 file changed, 5 insertions(+)
arch/arm/mach-k3/arm64/arm64-mmu.c +5 -0
@@ -41,3 +41,8 @@ struct mm_region k3_mem_map[K3_MEM_MAP_LEN] = { };struct mm_region *mem_map = k3_mem_map; + +u64 get_page_table_size (void ) +{ +return SZ_128K ; +}
Signature · 1 line
2.51.0
Quoted 10 lines · Anshul Dalal
In U-Boot, TI only provides a single memory map for all k3 platforms, this does not scale for devices where atf and optee lie outside the range
0x80000000
-0x80080000and0x9e780000-0xa0000000respectively.There are also issues for devices with < 2GiB of memory (eg am62SiP with 512MiB of RAM) as the maximum size for the first DRAM bank is hardcoded to 2GiB in the current memory map. Furthermore the second DRAM bank is mapped even for devices that only have a single bank.
[...]
Applied to u-boot/master, thanks!
[01/11] mach-k3: use minimal memory map for all K3
commit: 4673c8dcb48d6b14070a4819ee4a6277a0f2daa6
[02/11] mach-k3: use custom enable_cache
commit: e55e57d240e0bdef604efca5b386978b1c70d513
[03/11] arm: armv8: mmu: export mmu_setup
commit: 567a683e8ced54c3ffa53f62a4ed7e535268c6f2
[04/11] arm: armv8: invalidate dcache entries on dcache_enable
commit: 9ebdbbc43e5fb5841d85ec7ebcb1dbf07f4c87b0
[05/11] arm: armv8: mmu: add mem_map_from_dram_banks
commit: fe2647f2a0d4e4df5d47fdf068aeb2a1edd3d533
[06/11] mach-k3: map all banks using mem_map_from_dram_banks
commit: f1c694b8fddece279cdd103ae4009bf25345d8e4
[07/11] arm: armv8: mmu: add mmu_unmap_reserved_mem
commit: 856480eef0a25dde339cce6d1889efdc836c6be8
[08/11] spl: split spl_board_fixups to arch/board specific
commit: 16ffcff0283d2f10821bad7cbcf89003a86c0063
[09/11] mach-k3: add reserved memory fixups for next boot stage
commit: 859f68ed2ee7a0c1f88efaeb769202df01ef7a97
[10/11] mach-k3: add carveouts for TFA and optee
commit: 4f5285f0e640c3e9b40420ba3b11358446646b11
[11/11] arm: mach-k3: reserve space for page table entries
commit: 342fd918b1bf2f00f52216297d5fe22272e8c7caSignature · 1 line
Tom
Hello Anshul,
Quoted 11 lines · Anshul Dalal
Hi all,
In U-Boot, TI only provides a single memory map for all k3 platforms, this does not scale for devices where atf and optee lie outside the range
0x80000000
-0x80080000and0x9e780000-0xa0000000respectively.There are also issues for devices with < 2GiB of memory (eg am62SiP with 512MiB of RAM) as the maximum size for the first DRAM bank is hardcoded to 2GiB in the current memory map. Furthermore the second DRAM bank is mapped even for devices that only have a single bank.
Quoted 8 lines
Therefore this patch set adds the required functionality to create the MMU table at runtime based on the device-tree.
The patch set has been build tested on all effected platforms but boot-tested only on TI's K3 EVMs, the beagleplay and phytec's phycore-am6* platforms.
The following effected boards have not been boot tested:
- verdin-am62
it seems that this series introduce a regression on verdin-am62, I have not done a bi-sect yet, but we run daily build of U-Boot master and the regressions seems to have started when this patch series was merged.
On verdin-am62 we detect the RAM size at run-time, see board/toradex/verdin-am62/verdin-am62.c:dram_init(), and now we always get 2GiB even for modules with only 512MB or 1024MB of memory.
I'll try to find sometime to bi-sect this, but maybe you already have some suggestion on what to look for.
Thanks
Francesco
Hi Francesco,
Quoted 23 lines · Francesco Dolcini
Hello Anshul,
Quoted 11 lines · Anshul Dalal
Hi all,
In U-Boot, TI only provides a single memory map for all k3 platforms, this does not scale for devices where atf and optee lie outside the range
0x80000000
-0x80080000and0x9e780000-0xa0000000respectively.There are also issues for devices with < 2GiB of memory (eg am62SiP with 512MiB of RAM) as the maximum size for the first DRAM bank is hardcoded to 2GiB in the current memory map. Furthermore the second DRAM bank is mapped even for devices that only have a single bank.
Quoted 8 lines
Therefore this patch set adds the required functionality to create the MMU table at runtime based on the device-tree.
The patch set has been build tested on all effected platforms but boot-tested only on TI's K3 EVMs, the beagleplay and phytec's phycore-am6* platforms.
The following effected boards have not been boot tested:
- verdin-am62
Quoted 9 lines
it seems that this series introduce a regression on verdin-am62, I have not done a bi-sect yet, but we run daily build of U-Boot master and the regressions seems to have started when this patch series was merged.
On verdin-am62 we detect the RAM size at run-time, see board/toradex/verdin-am62/verdin-am62.c:dram_init(), and now we always get 2GiB even for modules with only 512MB or 1024MB of memory.
This patch series modified the behavior of enable_caches to configure the memory map of the device as per the device-tree instead of using a static map for all of K3.
The issue with verdin-am62 seems to be that while you do properly configure gd->ram_size in your dram_init, the '/memory' node of the device-tree remains unchanged with the outdated 2GiB size.
You could try updating the fdt's memory size to the correct value in dram_init and see if that fixes the problem.
Regards,
Anshul
Quoted 44 lines · Anshul Dalal
Hi Francesco,
Quoted 23 lines · Francesco Dolcini
Hello Anshul,
Quoted 11 lines · Anshul Dalal
Hi all,
In U-Boot, TI only provides a single memory map for all k3 platforms, this does not scale for devices where atf and optee lie outside the range
0x80000000
-0x80080000and0x9e780000-0xa0000000respectively.There are also issues for devices with < 2GiB of memory (eg am62SiP with 512MiB of RAM) as the maximum size for the first DRAM bank is hardcoded to 2GiB in the current memory map. Furthermore the second DRAM bank is mapped even for devices that only have a single bank.
Quoted 8 lines
Therefore this patch set adds the required functionality to create the MMU table at runtime based on the device-tree.
The patch set has been build tested on all effected platforms but boot-tested only on TI's K3 EVMs, the beagleplay and phytec's phycore-am6* platforms.
The following effected boards have not been boot tested:
- verdin-am62Quoted 9 lines
it seems that this series introduce a regression on verdin-am62, I have not done a bi-sect yet, but we run daily build of U-Boot master and the regressions seems to have started when this patch series was merged.
On verdin-am62 we detect the RAM size at run-time, see board/toradex/verdin-am62/verdin-am62.c:dram_init(), and now we always get 2GiB even for modules with only 512MB or 1024MB of memory.
This patch series modified the behavior of
enable_cachesto configure the memory map of the device as per the device-tree instead of using a static map for all of K3.The issue with verdin-am62 seems to be that while you do properly configure
gd->ram_sizein yourdram_init, the '/memory' node of the device-tree remains unchanged with the outdated 2GiB size.
Quoted 5 lines
You could try updating the fdt's memory size to the correct value in
dram_initand see if that fixes the problem.Regards,
Anshul
Hello Anshul,
I was bisecting the series, and I can confirm that the commit "mach-k3: map all banks using mem_map_from_dram_banks" introduces the regression.
Given that initcall_run_f() calls dram_init_banksize(), and after relocation board_init_r() calls enable_caches() (call stack: board_init_r() -> initcall_run_r() -> initr_caches() -> enable_caches()), I would expect that enable_caches() should not override the bank sizes previously configured. Currently, however, enable_caches() introduces this side effect.
Wouldn’t it make more sense to call fdtdec_setup_memory_banksize() in the default dram_init_banksize() (in arch/arm/mach-k3/k3-ddr.c) and avoid calling it again in mem_map_from_dram_banks()?
Regards,
Emanuele
Quoted 61 lines · Emanuele Ghidoli
Quoted 44 lines · Anshul Dalal
Hi Francesco,
Quoted 23 lines · Francesco Dolcini
Hello Anshul,
Quoted 11 lines · Anshul Dalal
Hi all,
In U-Boot, TI only provides a single memory map for all k3 platforms, this does not scale for devices where atf and optee lie outside the range
0x80000000
-0x80080000and0x9e780000-0xa0000000respectively.There are also issues for devices with < 2GiB of memory (eg am62SiP with 512MiB of RAM) as the maximum size for the first DRAM bank is hardcoded to 2GiB in the current memory map. Furthermore the second DRAM bank is mapped even for devices that only have a single bank.
Quoted 8 lines
Therefore this patch set adds the required functionality to create the MMU table at runtime based on the device-tree.
The patch set has been build tested on all effected platforms but boot-tested only on TI's K3 EVMs, the beagleplay and phytec's phycore-am6* platforms.
The following effected boards have not been boot tested:
- verdin-am62Quoted 9 lines
it seems that this series introduce a regression on verdin-am62, I have not done a bi-sect yet, but we run daily build of U-Boot master and the regressions seems to have started when this patch series was merged.
On verdin-am62 we detect the RAM size at run-time, see board/toradex/verdin-am62/verdin-am62.c:dram_init(), and now we always get 2GiB even for modules with only 512MB or 1024MB of memory.
This patch series modified the behavior of
enable_cachesto configure the memory map of the device as per the device-tree instead of using a static map for all of K3.The issue with verdin-am62 seems to be that while you do properly configure
gd->ram_sizein yourdram_init, the '/memory' node of the device-tree remains unchanged with the outdated 2GiB size.Quoted 5 lines
You could try updating the fdt's memory size to the correct value in
dram_initand see if that fixes the problem.Regards,
AnshulHello Anshul,
I was bisecting the series, and I can confirm that the commit "mach-k3: map all banks using
mem_map_from_dram_banks" introduces the regression.Given that
initcall_run_f()callsdram_init_banksize(), and after relocationboard_init_r()callsenable_caches()(call stack:board_init_r()->initcall_run_r()->initr_caches()->enable_caches()), I would expect thatenable_caches()should not override the bank sizes previously configured. Currently, however,enable_caches()introduces this side effect.
Quoted 4 lines
Wouldn’t it make more sense to call
fdtdec_setup_memory_banksize()in the defaultdram_init_banksize()(inarch/arm/mach-k3/k3-ddr.c) and avoid calling it again inmem_map_from_dram_banks()?
We could follow that order too but that would makes a call to mem_map_from_dram_banks dependent on gd->bd->di_dram being correctly populated. I had assumed whoever calls mem_map_from_dram_banks had made sure to properly fixup the memory node of the fdt.
Given that it seems like the root cause of the problem is with the U-Boot's device-tree not having the correct memory node, we could add a call to fixup_memory_node (arch/arm/mach-k3/k3-ddr.c) from A53 SPL to ensure the memory can be queried stright from the device-tree once we do get to U-Boot proper.
--- a/board/toradex/verdin-am62/verdin-am62.c
+++ b/board/toradex/verdin-am62/verdin-am62.c
@@ -46,6 +46,13 @@ int dram_init_banksize(void)
return ret;
}
+#ifdef CONFIG_XPL_BUILD
+void spl_perform_board_fiups(struct spl_image_info *spl_image)
+{
+ fixup_memory_node(spl_image);
+}
+#endif
+
/*
* Avoid relocated U-Boot clash with Linux reserved-memory on 512 MB SoM
*/If you could get to U-Boot prompt with the above diff, could you share the output of the 'meminfo' command with the following configs added:
CONFIG_CMD_MEMINFO=y
CONFIG_CMD_MEMINFO_MAP=y
Though so far, I have been unsuccessful in my attempts to reproduce a boot failure on our own 512MiB platforms (AM62x SiP). Could you share the boot logs with '#define DEBUG' in common/board_r.c, common/board_f.c and mach-k3/common.c to help further narrow down the issue.
Regards,
Anshul
Quoted 102 lines · Anshul Dalal
Quoted 61 lines · Emanuele Ghidoli
Quoted 44 lines · Anshul Dalal
Hi Francesco,
Quoted 23 lines · Francesco Dolcini
Hello Anshul,
Quoted 11 lines · Anshul Dalal
Hi all,
In U-Boot, TI only provides a single memory map for all k3 platforms, this does not scale for devices where atf and optee lie outside the range
0x80000000
-0x80080000and0x9e780000-0xa0000000respectively.There are also issues for devices with < 2GiB of memory (eg am62SiP with 512MiB of RAM) as the maximum size for the first DRAM bank is hardcoded to 2GiB in the current memory map. Furthermore the second DRAM bank is mapped even for devices that only have a single bank.
Quoted 8 lines
Therefore this patch set adds the required functionality to create the MMU table at runtime based on the device-tree.
The patch set has been build tested on all effected platforms but boot-tested only on TI's K3 EVMs, the beagleplay and phytec's phycore-am6* platforms.
The following effected boards have not been boot tested:
- verdin-am62Quoted 9 lines
it seems that this series introduce a regression on verdin-am62, I have not done a bi-sect yet, but we run daily build of U-Boot master and the regressions seems to have started when this patch series was merged.
On verdin-am62 we detect the RAM size at run-time, see board/toradex/verdin-am62/verdin-am62.c:dram_init(), and now we always get 2GiB even for modules with only 512MB or 1024MB of memory.
This patch series modified the behavior of
enable_cachesto configure the memory map of the device as per the device-tree instead of using a static map for all of K3.The issue with verdin-am62 seems to be that while you do properly configure
gd->ram_sizein yourdram_init, the '/memory' node of the device-tree remains unchanged with the outdated 2GiB size.Quoted 5 lines
You could try updating the fdt's memory size to the correct value in
dram_initand see if that fixes the problem.Regards,
AnshulHello Anshul,
I was bisecting the series, and I can confirm that the commit "mach-k3: map all banks using
mem_map_from_dram_banks" introduces the regression.Given that
initcall_run_f()callsdram_init_banksize(), and after relocationboard_init_r()callsenable_caches()(call stack:board_init_r()->initcall_run_r()->initr_caches()->enable_caches()), I would expect thatenable_caches()should not override the bank sizes previously configured. Currently, however,enable_caches()introduces this side effect.Quoted 4 lines
Wouldn’t it make more sense to call
fdtdec_setup_memory_banksize()in the defaultdram_init_banksize()(inarch/arm/mach-k3/k3-ddr.c) and avoid calling it again inmem_map_from_dram_banks()?We could follow that order too but that would makes a call to
mem_map_from_dram_banksdependent ongd->bd->di_drambeing correctly populated. I had assumed whoever callsmem_map_from_dram_bankshad made sure to properly fixup the memory node of the fdt.Given that it seems like the root cause of the problem is with the U-Boot's device-tree not having the correct memory node, we could add a call to
fixup_memory_node(arch/arm/mach-k3/k3-ddr.c) from A53 SPL to ensure the memory can be queried stright from the device-tree once we do get to U-Boot proper.--- a/board/toradex/verdin-am62/verdin-am62.c +++ b/board/toradex/verdin-am62/verdin-am62.c @@ -46,6 +46,13 @@ int dram_init_banksize(void) return ret; } +#ifdef CONFIG_XPL_BUILD +void spl_perform_board_fiups(struct spl_image_info *spl_image) +{ + fixup_memory_node(spl_image); +} +#endif + /* * Avoid relocated U-Boot clash with Linux reserved-memory on 512 MB SoM */If you could get to U-Boot prompt with the above diff, could you share the output of the 'meminfo' command with the following configs added:
CONFIG_CMD_MEMINFO=y
CONFIG_CMD_MEMINFO_MAP=y
Quoted 7 lines
Though so far, I have been unsuccessful in my attempts to reproduce a boot failure on our own 512MiB platforms (AM62x SiP). Could you share the boot logs with '#define DEBUG' in
common/board_r.c,common/board_f.candmach-k3/common.cto help further narrow down the issue.Regards,
Anshul
Hello Anshul,
let me try to rephrase.
The enable_caches() function is not only enabling caches, but also updating the memory map.
It is not expected from the point of view of the caller and this side effect introduces a regression on our U-Boot, since the memory banks are already configured in dram_init().
I understand that updating the device tree, as you proposed, could work around the issue, but that does not really fix the root cause.
Having a function perform unexpected operations is the best way to introduce regressions now and bugs in the future.
We should either revert the patch or properly fix the issue.
Regards,
Emanuele
Quoted 113 lines · Emanuele Ghidoli
Quoted 102 lines · Anshul Dalal
Quoted 61 lines · Emanuele Ghidoli
Quoted 44 lines · Anshul Dalal
Hi Francesco,
Quoted 23 lines · Francesco Dolcini
Hello Anshul,
Quoted 11 lines · Anshul Dalal
Hi all,
In U-Boot, TI only provides a single memory map for all k3 platforms, this does not scale for devices where atf and optee lie outside the range
0x80000000
-0x80080000and0x9e780000-0xa0000000respectively.There are also issues for devices with < 2GiB of memory (eg am62SiP with 512MiB of RAM) as the maximum size for the first DRAM bank is hardcoded to 2GiB in the current memory map. Furthermore the second DRAM bank is mapped even for devices that only have a single bank.
Quoted 8 lines
Therefore this patch set adds the required functionality to create the MMU table at runtime based on the device-tree.
The patch set has been build tested on all effected platforms but boot-tested only on TI's K3 EVMs, the beagleplay and phytec's phycore-am6* platforms.
The following effected boards have not been boot tested:
- verdin-am62Quoted 9 lines
it seems that this series introduce a regression on verdin-am62, I have not done a bi-sect yet, but we run daily build of U-Boot master and the regressions seems to have started when this patch series was merged.
On verdin-am62 we detect the RAM size at run-time, see board/toradex/verdin-am62/verdin-am62.c:dram_init(), and now we always get 2GiB even for modules with only 512MB or 1024MB of memory.
This patch series modified the behavior of
enable_cachesto configure the memory map of the device as per the device-tree instead of using a static map for all of K3.The issue with verdin-am62 seems to be that while you do properly configure
gd->ram_sizein yourdram_init, the '/memory' node of the device-tree remains unchanged with the outdated 2GiB size.Quoted 5 lines
You could try updating the fdt's memory size to the correct value in
dram_initand see if that fixes the problem.Regards,
AnshulHello Anshul,
I was bisecting the series, and I can confirm that the commit "mach-k3: map all banks using
mem_map_from_dram_banks" introduces the regression.Given that
initcall_run_f()callsdram_init_banksize(), and after relocationboard_init_r()callsenable_caches()(call stack:board_init_r()->initcall_run_r()->initr_caches()->enable_caches()), I would expect thatenable_caches()should not override the bank sizes previously configured. Currently, however,enable_caches()introduces this side effect.Quoted 4 lines
Wouldn’t it make more sense to call
fdtdec_setup_memory_banksize()in the defaultdram_init_banksize()(inarch/arm/mach-k3/k3-ddr.c) and avoid calling it again inmem_map_from_dram_banks()?We could follow that order too but that would makes a call to
mem_map_from_dram_banksdependent ongd->bd->di_drambeing correctly populated. I had assumed whoever callsmem_map_from_dram_bankshad made sure to properly fixup the memory node of the fdt.Given that it seems like the root cause of the problem is with the U-Boot's device-tree not having the correct memory node, we could add a call to
fixup_memory_node(arch/arm/mach-k3/k3-ddr.c) from A53 SPL to ensure the memory can be queried stright from the device-tree once we do get to U-Boot proper.--- a/board/toradex/verdin-am62/verdin-am62.c +++ b/board/toradex/verdin-am62/verdin-am62.c @@ -46,6 +46,13 @@ int dram_init_banksize(void) return ret; } +#ifdef CONFIG_XPL_BUILD +void spl_perform_board_fiups(struct spl_image_info *spl_image) +{ + fixup_memory_node(spl_image); +} +#endif + /* * Avoid relocated U-Boot clash with Linux reserved-memory on 512 MB SoM */If you could get to U-Boot prompt with the above diff, could you share the output of the 'meminfo' command with the following configs added:
CONFIG_CMD_MEMINFO=y
CONFIG_CMD_MEMINFO_MAP=yQuoted 7 lines
Though so far, I have been unsuccessful in my attempts to reproduce a boot failure on our own 512MiB platforms (AM62x SiP). Could you share the boot logs with '#define DEBUG' in
common/board_r.c,common/board_f.candmach-k3/common.cto help further narrow down the issue.Regards,
Anshul
Quoted 8 lines
Hello Anshul,
let me try to rephrase.The
enable_caches()function is not only enabling caches, but also updating the memory map.
It is not expected from the point of view of the caller and this side effect introduces a regression on our U-Boot, since the memory banks are already configured indram_init().
Hi Emanuele,
Ah, I see your point. In that case mem_map_from_dram_banks should only rely on the gd with dram_init and dram_init_banksize taking care of configuring gd properly.
I'll send a fix as per your suggestion (moving fdtdec_setup_memory_banksize to the default dram_init_banksize) shortly.
Regards,
Anshul
Quoted 10 lines
I understand that updating the device tree, as you proposed, could work around the issue, but that does not really fix the root cause.
Having a function perform unexpected operations is the best way to introduce regressions now and bugs in the future.We should either revert the patch or properly fix the issue.
Regards,
Emanuele