Skip to contentSkip to conversation

arm64: Unify MMU code v3

PATCH New Alexander Graf 3 participants 18 messages Feb 26, 2016

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

v4 cover arm64: Unify MMU code v3

Howdy,

Currently on arm64 there is a big pile of mess when it comes to MMU support and page tables. Each board does its own little thing and the generic code is pretty dumb and nobody actually uses it.

This patch set tries to clean that up. After this series is applied, all boards except for the FSL Layerscape ones are converted to the new generic page table logic and have icache+dcache enabled.

The new code always uses 4k page size. It dynamically allocates 1G or 2M pages for ranges that fit. When a dcache attribute request comes in that requires a smaller granularity than our previous allocation could fulfill, pages get automatically split.

I have tested and verified the code works on HiKey (bare metal), vexpress64 (Foundation Model) and zynqmp (QEMU). The TX1 target is untested, but given the simplicity of the maps I doubt it'll break. ThunderX in theory should also work, but I haven't tested it. I would be very happy if people with access to those system could give the patch set a try.

With this we're a big step closer to a good base line for EFI payload support, since we can now just require that all boards always have dcache enabled.

I would also be incredibly happy if some Freescale people could look at their MMU code and try to unify it into the now cleaned up generic code. I don't think we're far off here.

Alex

v1 -> v2:

- Fix comment for create_table()
- Rework page table size calculation
- Move mmu tables into board files
- New patch: thunderx: Move mmu table into board file

v2 -> v3:

- Move PGTABLE_SIZE out of the ASSEMBLY section
- Run pte splitting code in separate page table
- Replace is_level in count_required_pts with more obvious pte_type
- New patch: arm64: Disable TTBR1 maps in EL1

v3 -> v4:

- Fix Lv3 PTEs that are always of type PTE_TYPE_LEVEL (fixes zynqmp)

Alexander Graf (11):
thunderx: Calculate TCR dynamically
arm64: Disable TTBR1 maps in EL1
arm64: Make full va map code more dynamic
thunderx: Move mmu table into board file
zymqmp: Replace home grown mmu code with generic table approach
tegra: Replace home grown mmu code with generic table approach
vexpress64: Add MMU tables
dwmmc: Increase retry timeout
hikey: Add MMU tables
arm64: Remove non-full-va map code
arm64: Only allow dcache disabled in SPL builds

 arch/arm/cpu/armv8/cache.S                     |  54 +++
 arch/arm/cpu/armv8/cache_v8.c                  | 551 +++++++++++++++++++------
 arch/arm/cpu/armv8/fsl-layerscape/cpu.c        |  37 +-
 arch/arm/cpu/armv8/zynqmp/cpu.c                | 217 +++-------
 arch/arm/include/asm/arch-fsl-layerscape/cpu.h |  94 ++---
 arch/arm/include/asm/armv8/mmu.h               | 123 ++----
 arch/arm/include/asm/global_data.h             |   7 +-
 arch/arm/include/asm/system.h                  |  12 +-
 arch/arm/mach-tegra/arm64-mmu.c                | 132 +-----
 board/armltd/vexpress64/vexpress64.c           |  21 +
 board/cavium/thunderx/thunderx.c               |  24 ++
 board/hisilicon/hikey/hikey.c                  |  21 +
 doc/README.arm64                               |  20 -
 drivers/mmc/dw_mmc.c                           |   2 +-
 include/configs/hikey.h                        |   4 +-
 include/configs/thunderx_88xx.h                |  30 --
 include/configs/vexpress_aemv8a.h              |   5 +-
 17 files changed, 738 insertions(+), 616 deletions(-)
Signature · 1 line
1.8.5.6
#1 lore raw

PATCH v4 1/11 thunderx: Calculate TCR dynamically

Based on the memory map we can determine a lot of hard coded fields of TCR, like the maximum VA and max PA we want to support. Calculate those dynamically to reduce the chance for pit falls.

---
 arch/arm/cpu/armv8/cache_v8.c    | 59 +++++++++++++++++++++++++++++++++++++++-
 arch/arm/include/asm/armv8/mmu.h |  6 +---
 include/configs/thunderx_88xx.h  |  3 --
 3 files changed, 59 insertions(+), 9 deletions(-)
arch/arm/cpu/armv8/cache_v8.c +58 -1
index 71f0020..9229532 100644
@@ -38,6 +38,58 @@ static struct mm_region mem_map[] = CONFIG_SYS_MEM_MAP;
 #define PTL1_ENTRIES CONFIG_SYS_PTL1_ENTRIES
 #define PTL2_ENTRIES CONFIG_SYS_PTL2_ENTRIES
 
+static u64 get_tcr(int el, u64 *pips, u64 *pva_bits)
+{
+	u64 max_addr = 0;
+	u64 ips, va_bits;
+	u64 tcr;
+	int i;
+
+	/* Find the largest address we need to support */
+	for (i = 0; i < ARRAY_SIZE(mem_map); i++)
+		max_addr = max(max_addr, mem_map[i].base + mem_map[i].size);
+
+	/* Calculate the maximum physical (and thus virtual) address */
+	if (max_addr > (1ULL << 44)) {
+		ips = 5;
+		va_bits = 48;
+	} else  if (max_addr > (1ULL << 42)) {
+		ips = 4;
+		va_bits = 44;
+	} else  if (max_addr > (1ULL << 40)) {
+		ips = 3;
+		va_bits = 42;
+	} else  if (max_addr > (1ULL << 36)) {
+		ips = 2;
+		va_bits = 40;
+	} else  if (max_addr > (1ULL << 32)) {
+		ips = 1;
+		va_bits = 36;
+	} else {
+		ips = 0;
+		va_bits = 32;
+	}
+
+	if (el == 1) {
+		tcr = TCR_EL1_RSVD | (ips << 32);
+	} else if (el == 2) {
+		tcr = TCR_EL2_RSVD | (ips << 16);
+	} else {
+		tcr = TCR_EL3_RSVD | (ips << 16);
+	}
+
+	/* PTWs cacheable, inner/outer WBWA and inner shareable */
+	tcr |= TCR_TG0_64K | TCR_SHARED_INNER | TCR_ORGN_WBWA | TCR_IRGN_WBWA;
+	tcr |= TCR_T0SZ(VA_BITS);
+
+	if (pips)
+		*pips = ips;
+	if (pva_bits)
+		*pva_bits = va_bits;
+
+	return tcr;
+}
+
 static void setup_pgtables(void)
 {
 	int l1_e, l2_e;
@@ -110,6 +162,10 @@ __weak void mmu_setup(void)
 	/* Set up page tables only on BSP */
 	if (coreid == BSP_COREID)
 		setup_pgtables();
+
+	el = current_el();
+	set_ttbr_tcr_mair(el, gd->arch.tlb_addr, get_tcr(el, NULL, NULL),
+			  MEMORY_ATTRIBUTES);
 #else
 	/* Setup an identity-mapping for all spaces */
 	for (i = 0; i < (PGTABLE_SIZE >> 3); i++) {
@@ -128,7 +184,6 @@ __weak void mmu_setup(void)
 		}
 	}
 
-#endif
 	/* load TTBR0 */
 	el = current_el();
 	if (el == 1) {
@@ -144,6 +199,8 @@ __weak void mmu_setup(void)
 				  TCR_EL3_RSVD | TCR_FLAGS | TCR_EL3_IPS_BITS,
 				  MEMORY_ATTRIBUTES);
 	}
+#endif
+
 	/* enable the mmu */
 	set_sctlr(get_sctlr() | CR_M);
 }
arch/arm/include/asm/armv8/mmu.h +1 -5
index 897f010..39ff745 100644
@@ -159,11 +159,6 @@
 #define TCR_EL1_IPS_BITS	(UL(3) << 32)	/* 42 bits physical address */
 #define TCR_EL2_IPS_BITS	(3 << 16)	/* 42 bits physical address */
 #define TCR_EL3_IPS_BITS	(3 << 16)	/* 42 bits physical address */
-#else
-#define TCR_EL1_IPS_BITS	CONFIG_SYS_TCR_EL1_IPS_BITS
-#define TCR_EL2_IPS_BITS	CONFIG_SYS_TCR_EL2_IPS_BITS
-#define TCR_EL3_IPS_BITS	CONFIG_SYS_TCR_EL3_IPS_BITS
-#endif
 
 /* PTWs cacheable, inner/outer WBWA and inner shareable */
 #define TCR_FLAGS		(TCR_TG0_64K |		\
@@ -171,6 +166,7 @@
 				TCR_ORGN_WBWA |		\
 				TCR_IRGN_WBWA |		\
 				TCR_T0SZ(VA_BITS))
+#endif
 
 #define TCR_EL1_RSVD		(1 << 31)
 #define TCR_EL2_RSVD		(1 << 31 | 1 << 23)
include/configs/thunderx_88xx.h +0 -3
index cece4dd..b9f93ad 100644
@@ -50,9 +50,6 @@
 #define CONFIG_SYS_PGTABLE_SIZE		\
 	((CONFIG_SYS_PTL1_ENTRIES + \
 	  CONFIG_SYS_MEM_MAP_SIZE * CONFIG_SYS_PTL2_ENTRIES) * 8)
-#define CONFIG_SYS_TCR_EL1_IPS_BITS	(5UL << 32)
-#define CONFIG_SYS_TCR_EL2_IPS_BITS	(5 << 16)
-#define CONFIG_SYS_TCR_EL3_IPS_BITS	(5 << 16)
 
 /* Link Definitions */
 #define CONFIG_SYS_TEXT_BASE		0x00500000
Signature · 1 line
1.8.5.6
#2 lore raw

PATCH v4 1/11 thunderx: Calculate TCR dynamically

Hi,

Quoted 10 lines · Alexander Graf

Based on the memory map we can determine a lot of hard coded fields of TCR, like the maximum VA and max PA we want to support. Calculate those dynamically to reduce the chance for pit falls.

---
 arch/arm/cpu/armv8/cache_v8.c    | 59 +++++++++++++++++++++++++++++++++++++++-
 arch/arm/include/asm/armv8/mmu.h |  6 +---
 include/configs/thunderx_88xx.h  |  3 --
 3 files changed, 59 insertions(+), 9 deletions(-)

This approach (extern variable) looks good to me.

I didn't know ARRAY_SIZE() worked on external variables, so I leaned something also.

Regards,
Simon

#3 lore raw

PATCH v4 1/11 thunderx: Calculate TCR dynamically

Quoted 18 lines

Am 26.02.2016 um 03:51 schrieb Simon Glass <[email protected]>:

Hi,

Quoted 11 lines · Alexander Graf

Based on the memory map we can determine a lot of hard coded fields of TCR, like the maximum VA and max PA we want to support. Calculate those dynamically to reduce the chance for pit falls.

---

arch/arm/cpu/armv8/cache_v8.c    | 59 +++++++++++++++++++++++++++++++++++++++-
arch/arm/include/asm/armv8/mmu.h |  6 +---
include/configs/thunderx_88xx.h  |  3 --
3 files changed, 59 insertions(+), 9 deletions(-)
Quoted 6 lines

This approach (extern variable) looks good to me.

I didn't know ARRAY_SIZE() worked on external variables, so I leaned something also.

It doesn't. Could you please point me to a case where I'm still using ARRAY_SIZE in v4? I thought I had moved everything to null-terminating entries.

Alex

#4 lore raw

PATCH v4 1/11 thunderx: Calculate TCR dynamically

Hi Alexander,

Quoted 18 lines · Alexander Graf
Quoted 18 lines

Am 26.02.2016 um 03:51 schrieb Simon Glass <[email protected]>:

Hi,

Quoted 11 lines · Alexander Graf

Based on the memory map we can determine a lot of hard coded fields of TCR, like the maximum VA and max PA we want to support. Calculate those dynamically to reduce the chance for pit falls.

---

arch/arm/cpu/armv8/cache_v8.c    | 59 +++++++++++++++++++++++++++++++++++++++-
arch/arm/include/asm/armv8/mmu.h |  6 +---
include/configs/thunderx_88xx.h  |  3 --
3 files changed, 59 insertions(+), 9 deletions(-)
Quoted 8 lines
Quoted 6 lines

This approach (extern variable) looks good to me.

I didn't know ARRAY_SIZE() worked on external variables, so I leaned something also.

It doesn't. Could you please point me to a case where I'm still using ARRAY_SIZE in v4? I thought I had moved everything to null-terminating entries.

I imagined some gcc linker sizeof() magic. Oh well.

See near the top of get_tcr() - the patch on this thread.

Regards,
Simon

#5 lore raw

PATCH v4 1/11 thunderx: Calculate TCR dynamically

Quoted 21 lines · Simon Glass

Hi Alexander,

Quoted 18 lines · Alexander Graf
Quoted 18 lines

Am 26.02.2016 um 03:51 schrieb Simon Glass <[email protected]>:

Hi,

Quoted 11 lines · Alexander Graf

Based on the memory map we can determine a lot of hard coded fields of TCR, like the maximum VA and max PA we want to support. Calculate those dynamically to reduce the chance for pit falls.

---

arch/arm/cpu/armv8/cache_v8.c    | 59 +++++++++++++++++++++++++++++++++++++++-
arch/arm/include/asm/armv8/mmu.h |  6 +---
include/configs/thunderx_88xx.h  |  3 --
3 files changed, 59 insertions(+), 9 deletions(-)

This approach (extern variable) looks good to me.

Quoted 8 lines
Quoted 5 lines
Quoted 4 lines

I didn't know ARRAY_SIZE() worked on external variables, so I leaned something also.

It doesn't. Could you please point me to a case where I'm still using ARRAY_SIZE in v4? I thought I had moved everything to null-terminating entries.

I imagined some gcc linker sizeof() magic. Oh well.

See near the top of get_tcr() - the patch on this thread.

In 1/11 the mem_map is still a locally set array which gets preinitialized to a constant array. That one is supported by ARRAY_SIZE.

Later in "thunderx: Move mmu table into board file" I move the mem_map variable into board files and change all ARRAY_SIZE instances to loops that loop until they hit an empty entry. This is mostly to make review and bisectability easier, since that way only few things change at a time.

Alex

#6 lore raw

PATCH v4 1/11 thunderx: Calculate TCR dynamically

Hi Alex,

Quoted 40 lines · Alexander Graf
Quoted 30 lines · Simon Glass

Hi Alexander,

Quoted 26 lines · Alexander Graf
Quoted 25 lines

Am 26.02.2016 um 03:51 schrieb Simon Glass <[email protected]>:

Hi,

Quoted 5 lines · Alexander Graf

Based on the memory map we can determine a lot of hard coded fields of TCR, like the maximum VA and max PA we want to support. Calculate those dynamically to reduce the chance for pit falls.

Quoted 7 lines

---
arch/arm/cpu/armv8/cache_v8.c | 59
+++++++++++++++++++++++++++++++++++++++-

arch/arm/include/asm/armv8/mmu.h |  6 +---
include/configs/thunderx_88xx.h  |  3 --
3 files changed, 59 insertions(+), 9 deletions(-)

This approach (extern variable) looks good to me.

I didn't know ARRAY_SIZE() worked on external variables, so I leaned something also.

Quoted 7 lines
Quoted 3 lines

It doesn't. Could you please point me to a case where I'm still using ARRAY_SIZE in v4? I thought I had moved everything to null-terminating entries.

I imagined some gcc linker sizeof() magic. Oh well.

See near the top of get_tcr() - the patch on this thread.

Quoted 7 lines

In 1/11 the mem_map is still a locally set array which gets preinitialized to a constant array. That one is supported by ARRAY_SIZE.

Later in "thunderx: Move mmu table into board file" I move the mem_map variable into board files and change all ARRAY_SIZE instances to loops that loop until they hit an empty entry. This is mostly to make review and bisectability easier, since that way only few things change at a time.

OK, makes sense. Sorry I haven't received it very closely but you seem to have got plenty of input.

Regards,
Simon

#7 lore raw

PATCH v4 2/11 arm64: Disable TTBR1 maps in EL1

In reply to Alexander Graf

When running in EL1, AArch64 knows two page table maps. One with addresses that start with all zeros (TTBR0) and one with addresses that start with all ones (TTBR1).

In U-Boot we don't care about the high up maps, so just disable them to ensure we don't walk an invalid page table by accident.

---
 arch/arm/cpu/armv8/cache_v8.c    | 2 +-
 arch/arm/include/asm/armv8/mmu.h | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)
arch/arm/cpu/armv8/cache_v8.c +1 -1
index 9229532..d92f2d1 100644
@@ -71,7 +71,7 @@ static u64 get_tcr(int el, u64 *pips, u64 *pva_bits)
 	}
 
 	if (el == 1) {
-		tcr = TCR_EL1_RSVD | (ips << 32);
+		tcr = TCR_EL1_RSVD | (ips << 32) | TCR_EPD1_DISABLE;
 	} else if (el == 2) {
 		tcr = TCR_EL2_RSVD | (ips << 16);
 	} else {
arch/arm/include/asm/armv8/mmu.h +1 -0
index 39ff745..0080ae6 100644
@@ -154,6 +154,7 @@
 #define TCR_TG0_4K		(0 << 14)
 #define TCR_TG0_64K		(1 << 14)
 #define TCR_TG0_16K		(2 << 14)
+#define TCR_EPD1_DISABLE	(1 << 23)
 
 #ifndef CONFIG_SYS_FULL_VA
 #define TCR_EL1_IPS_BITS	(UL(3) << 32)	/* 42 bits physical address */
Signature · 1 line
1.8.5.6
#8 lore raw

PATCH v4 3/11 arm64: Make full va map code more dynamic

In reply to Alexander Graf

The idea to generate our pages tables from an array of memory ranges is very sound. However, instead of hard coding the code to create up to 2 levels of 64k granule page tables, we really should just create normal 4k page tables that allow us to set caching attributes on 2M or 4k level later on.

So this patch moves the full_va mapping code to 4k page size and makes it fully flexible to dynamically create as many levels as necessary for a map (including dynamic 1G/2M pages). It also adds support to dynamically split a large map into smaller ones when some code wants to set dcache attributes.

With all this in place, there is very little reason to create your own page tables in board specific files.

---

v1 -> v2:

- Fix comment for create_table()
- Rework page table size calculation
- s/DPRINTF/debug/g
- Improve panic messages

v2 -> v3:

- Move PGTABLE_SIZE out of the ASSEMBLY section
- Run pte splitting code in separate page table
- Replace is_level in count_required_pts with more obvious pte_type

v3 -> v4:

- Lv3 PTEs are always of type table

---
 arch/arm/cpu/armv8/cache.S         |  54 +++++
 arch/arm/cpu/armv8/cache_v8.c      | 453 ++++++++++++++++++++++++++++++++-----
 arch/arm/include/asm/armv8/mmu.h   |  68 +++---
 arch/arm/include/asm/global_data.h |   5 +-
 arch/arm/include/asm/system.h      |  14 +-
 include/configs/thunderx_88xx.h    |  14 +-
 6 files changed, 500 insertions(+), 108 deletions(-)
arch/arm/cpu/armv8/cache.S +54 -0
index ab8c089..a9f4fec 100644
@@ -10,6 +10,7 @@
 #include <asm-offsets.h>
 #include <config.h>
 #include <asm/macro.h>
+#include <asm/system.h>
 #include <linux/linkage.h>
 
 /*
@@ -160,3 +161,56 @@ ENTRY(__asm_flush_l3_cache)
 	ret
 ENDPROC(__asm_flush_l3_cache)
 	.weak	__asm_flush_l3_cache
+
+/*
+ * void __asm_switch_ttbr(ulong new_ttbr)
+ *
+ * Safely switches to a new page table.
+ */
+ENTRY(__asm_switch_ttbr)
+	/* x2 = SCTLR (alive throghout the function) */
+	switch_el x4, 3f, 2f, 1f
+3:	mrs	x2, sctlr_el3
+	b	0f
+2:	mrs	x2, sctlr_el2
+	b	0f
+1:	mrs	x2, sctlr_el1
+0:
+
+	/* Unset CR_M | CR_C | CR_I from SCTLR to disable all caches */
+	movn	x1, #(CR_M | CR_C | CR_I)
+	and	x1, x2, x1
+	switch_el x4, 3f, 2f, 1f
+3:	msr	sctlr_el3, x1
+	b	0f
+2:	msr	sctlr_el2, x1
+	b	0f
+1:	msr	sctlr_el1, x1
+0:	isb
+
+	/* This call only clobbers x30 (lr) and x9 (unused) */
+	mov	x3, x30
+	bl	__asm_invalidate_tlb_all
+
+	/* From here on we're running safely with caches disabled */
+
+	/* Set TTBR to our first argument */
+	switch_el x4, 3f, 2f, 1f
+3:	msr	ttbr0_el3, x0
+	b	0f
+2:	msr	ttbr0_el2, x0
+	b	0f
+1:	msr	ttbr0_el1, x0
+0:	isb
+
+	/* Restore original SCTLR and thus enable caches again */
+	switch_el x4, 3f, 2f, 1f
+3:	msr	sctlr_el3, x2
+	b	0f
+2:	msr	sctlr_el2, x2
+	b	0f
+1:	msr	sctlr_el1, x2
+0:	isb
+
+	ret	x3
+ENDPROC(__asm_switch_ttbr)
arch/arm/cpu/armv8/cache_v8.c +398 -55

499 lines not on this page · raw message

arch/arm/include/asm/armv8/mmu.h +31 -37

95 lines not on this page · raw message

arch/arm/include/asm/global_data.h +3 -2
index dcfa098..259daa1 100644
@@ -38,10 +38,11 @@ struct arch_global_data {
 	unsigned long long timer_reset_value;
 #if !(defined(CONFIG_SYS_ICACHE_OFF) && defined(CONFIG_SYS_DCACHE_OFF))
 	unsigned long tlb_addr;
+	unsigned long tlb_size;
 #if defined(CONFIG_SYS_FULL_VA)
-	unsigned long pmd_addr[CONFIG_SYS_PTL1_ENTRIES];
+	unsigned long tlb_fillptr;
+	unsigned long tlb_emerg;
 #endif
-	unsigned long tlb_size;
 #endif
 
 #ifdef CONFIG_OMAP_COMMON
arch/arm/include/asm/system.h +8 -6
index 026e7ef..9b1cbf2 100644
@@ -17,18 +17,19 @@
 #define CR_WXN		(1 << 19)	/* Write Permision Imply XN	*/
 #define CR_EE		(1 << 25)	/* Exception (Big) Endian	*/
 
-#ifndef CONFIG_SYS_FULL_VA
-#define PGTABLE_SIZE	(0x10000)
-#else
-#define PGTABLE_SIZE	CONFIG_SYS_PGTABLE_SIZE
-#endif
-
 /* 2MB granularity */
 #define MMU_SECTION_SHIFT	21
 #define MMU_SECTION_SIZE	(1 << MMU_SECTION_SHIFT)
 
 #ifndef __ASSEMBLY__
 
+#ifndef CONFIG_SYS_FULL_VA
+#define PGTABLE_SIZE	(0x10000)
+#else
+u64 get_page_table_size(void);
+#define PGTABLE_SIZE	get_page_table_size()
+#endif
+
 enum dcache_option {
 	DCACHE_OFF = 0x3,
 };
@@ -97,6 +98,7 @@ void __asm_flush_dcache_range(u64 start, u64 end);
 void __asm_invalidate_tlb_all(void);
 void __asm_invalidate_icache_all(void);
 int __asm_flush_l3_cache(void);
+void __asm_switch_ttbr(u64 new_ttbr);
 
 void armv8_switch_to_el2(void);
 void armv8_switch_to_el1(void);
include/configs/thunderx_88xx.h +6 -8
index b9f93ad..20b25f7 100644
@@ -22,21 +22,19 @@
 
 #define MEM_BASE			0x00500000
 
-#define CONFIG_COREID_MASK             0xffffff
-
 #define CONFIG_SYS_FULL_VA
 
 #define CONFIG_SYS_LOWMEM_BASE		MEM_BASE
 
 #define CONFIG_SYS_MEM_MAP		{{0x000000000000UL, 0x40000000000UL, \
-					  PTL2_MEMTYPE(MT_NORMAL) |	     \
-					  PTL2_BLOCK_NON_SHARE},	     \
+					  PTE_BLOCK_MEMTYPE(MT_NORMAL) |     \
+					  PTE_BLOCK_NON_SHARE},	     \
 					 {0x800000000000UL, 0x40000000000UL, \
-					  PTL2_MEMTYPE(MT_DEVICE_NGNRNE) |   \
-					  PTL2_BLOCK_NON_SHARE},	     \
+					  PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) | \
+					  PTE_BLOCK_NON_SHARE},	     \
 					 {0x840000000000UL, 0x40000000000UL, \
-					  PTL2_MEMTYPE(MT_DEVICE_NGNRNE) |   \
-					  PTL2_BLOCK_NON_SHARE},	     \
+					  PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) | \
+					  PTE_BLOCK_NON_SHARE},	     \
 					}
 
 #define CONFIG_SYS_MEM_MAP_SIZE		3
Signature · 1 line
1.8.5.6
#9 lore raw

PATCH v4 4/11 thunderx: Move mmu table into board file

In reply to Alexander Graf

The MMU range table can vary depending on things we may only find out at runtime. While the very simple ThunderX variant does not change, other boards will, so move the definition from a static entry in a header file to the board file.

---
 arch/arm/cpu/armv8/cache_v8.c    |  8 +++-----
 arch/arm/include/asm/armv8/mmu.h |  2 ++
 board/cavium/thunderx/thunderx.c | 24 ++++++++++++++++++++++++
 include/configs/thunderx_88xx.h  | 11 -----------
 4 files changed, 29 insertions(+), 16 deletions(-)
arch/arm/cpu/armv8/cache_v8.c +3 -5
index 73628c9..55c6f2f 100644
@@ -36,8 +36,6 @@ DECLARE_GLOBAL_DATA_PTR;
  */
 
 #ifdef CONFIG_SYS_FULL_VA
-static struct mm_region mem_map[] = CONFIG_SYS_MEM_MAP;
-
 static u64 get_tcr(int el, u64 *pips, u64 *pva_bits)
 {
 	u64 max_addr = 0;
@@ -46,7 +44,7 @@ static u64 get_tcr(int el, u64 *pips, u64 *pva_bits)
 	int i;
 
 	/* Find the largest address we need to support */
-	for (i = 0; i < ARRAY_SIZE(mem_map); i++)
+	for (i = 0; mem_map[i].size || mem_map[i].attrs; i++)
 		max_addr = max(max_addr, mem_map[i].base + mem_map[i].size);
 
 	/* Calculate the maximum physical (and thus virtual) address */
@@ -266,7 +264,7 @@ static int count_required_pts(u64 addr, int level, u64 maxaddr)
 	int i;
 	enum pte_type pte_type = PTE_INVAL;
 
-	for (i = 0; i < ARRAY_SIZE(mem_map); i++) {
+	for (i = 0; mem_map[i].size || mem_map[i].attrs; i++) {
 		struct mm_region *map = &mem_map[i];
 		u64 start = map->base;
 		u64 end = start + map->size;
@@ -364,7 +362,7 @@ static void setup_pgtables(void)
 	create_table();
 
 	/* Now add all MMU table entries one after another to the table */
-	for (i = 0; i < ARRAY_SIZE(mem_map); i++)
+	for (i = 0; mem_map[i].size || mem_map[i].attrs; i++)
 		add_map(&mem_map[i]);
 
 	/* Create the same thing once more for our emergency page table */
arch/arm/include/asm/armv8/mmu.h +2 -0
index 1c490dc..06126c8 100644
@@ -203,6 +203,8 @@ struct mm_region {
 	u64 size;
 	u64 attrs;
 };
+
+extern struct mm_region *mem_map;
 #endif
 
 #endif /* _ASM_ARMV8_MMU_H_ */
board/cavium/thunderx/thunderx.c +24 -0
index b926767..9131a38 100644
@@ -10,6 +10,7 @@
 #include <linux/compiler.h>
 
 #include <cavium/atf.h>
+#include <asm/armv8/mmu.h>
 
 #if !CONFIG_IS_ENABLED(OF_CONTROL)
 #include <dm/platdata.h>
@@ -42,6 +43,29 @@ U_BOOT_DEVICE(thunderx_serial1) = {
 
 DECLARE_GLOBAL_DATA_PTR;
 
+static struct mm_region thunderx_mem_map[] = {
+	{
+		.base = 0x000000000000UL,
+		.size = 0x40000000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_NORMAL) | PTE_BLOCK_NON_SHARE,
+	}, {
+		.base = 0x800000000000UL,
+		.size = 0x40000000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
+			 PTE_BLOCK_NON_SHARE,
+	}, {
+		.base = 0x840000000000UL,
+		.size = 0x40000000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
+			 PTE_BLOCK_NON_SHARE,
+	}, {
+		/* List terminator */
+		0,
+	}
+};
+
+struct mm_region *mem_map = thunderx_mem_map;
+
 int board_init(void)
 {
 	return 0;
include/configs/thunderx_88xx.h +0 -11
index 20b25f7..64e4616 100644
@@ -26,17 +26,6 @@
 
 #define CONFIG_SYS_LOWMEM_BASE		MEM_BASE
 
-#define CONFIG_SYS_MEM_MAP		{{0x000000000000UL, 0x40000000000UL, \
-					  PTE_BLOCK_MEMTYPE(MT_NORMAL) |     \
-					  PTE_BLOCK_NON_SHARE},	     \
-					 {0x800000000000UL, 0x40000000000UL, \
-					  PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) | \
-					  PTE_BLOCK_NON_SHARE},	     \
-					 {0x840000000000UL, 0x40000000000UL, \
-					  PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) | \
-					  PTE_BLOCK_NON_SHARE},	     \
-					}
-
 #define CONFIG_SYS_MEM_MAP_SIZE		3
 
 #define CONFIG_SYS_VA_BITS		48
Signature · 1 line
1.8.5.6
#10 lore raw

PATCH v4 5/11 zymqmp: Replace home grown mmu code with generic table approach

In reply to Alexander Graf

Now that we have nice table driven page table creating code that gives us everything we need, move to that.

---

v1 -> v2:

- Move mmu tables into board file

---
 arch/arm/cpu/armv8/zynqmp/cpu.c | 217 +++++++++-------------------------------
 include/configs/xilinx_zynqmp.h |   2 +
 2 files changed, 50 insertions(+), 169 deletions(-)
arch/arm/cpu/armv8/zynqmp/cpu.c +48 -169
index c71f291..5dd3cd8 100644
@@ -8,6 +8,7 @@
 #include <common.h>
 #include <asm/arch/hardware.h>
 #include <asm/arch/sys_proto.h>
+#include <asm/armv8/mmu.h>
 #include <asm/io.h>
 
 #define ZYNQ_SILICON_VER_MASK	0xF000
@@ -15,6 +16,53 @@
 
 DECLARE_GLOBAL_DATA_PTR;
 
+static struct mm_region zynqmp_mem_map[] = {
+	{
+		.base = 0x0UL,
+		.size = 0x80000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_NORMAL) |
+			 PTE_BLOCK_INNER_SHARE
+	}, {
+		.base = 0x80000000UL,
+		.size = 0x70000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
+			 PTE_BLOCK_NON_SHARE |
+			 PTE_BLOCK_PXN | PTE_BLOCK_UXN
+	}, {
+		.base = 0xf8000000UL,
+		.size = 0x07e00000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
+			 PTE_BLOCK_NON_SHARE |
+			 PTE_BLOCK_PXN | PTE_BLOCK_UXN
+	}, {
+		.base = 0xffe00000UL,
+		.size = 0x00200000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_NORMAL) |
+			 PTE_BLOCK_INNER_SHARE
+	}, {
+		.base = 0x400000000UL,
+		.size = 0x200000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
+			 PTE_BLOCK_NON_SHARE |
+			 PTE_BLOCK_PXN | PTE_BLOCK_UXN
+	}, {
+		.base = 0x600000000UL,
+		.size = 0x800000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_NORMAL) |
+			 PTE_BLOCK_INNER_SHARE
+	}, {
+		.base = 0xe00000000UL,
+		.size = 0xf200000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
+			 PTE_BLOCK_NON_SHARE |
+			 PTE_BLOCK_PXN | PTE_BLOCK_UXN
+	}, {
+		/* List terminator */
+		0,
+	}
+};
+struct mm_region *mem_map = zynqmp_mem_map;
+
 static unsigned int zynqmp_get_silicon_version_secure(void)
 {
 	u32 ver;
@@ -44,172 +92,3 @@ unsigned int zynqmp_get_silicon_version(void)
 
 	return ZYNQMP_CSU_VERSION_SILICON;
 }
-
-#ifndef CONFIG_SYS_DCACHE_OFF
-#include <asm/armv8/mmu.h>
-
-#define SECTION_SHIFT_L1	30UL
-#define SECTION_SHIFT_L2	21UL
-#define BLOCK_SIZE_L0		0x8000000000UL
-#define BLOCK_SIZE_L1		(1 << SECTION_SHIFT_L1)
-#define BLOCK_SIZE_L2		(1 << SECTION_SHIFT_L2)
-
-#define TCR_TG1_4K		(1 << 31)
-#define TCR_EPD1_DISABLE	(1 << 23)
-#define ZYNQMO_VA_BITS		40
-#define ZYNQMP_TCR		TCR_TG1_4K | \
-				TCR_EPD1_DISABLE | \
-				TCR_SHARED_OUTER | \
-				TCR_SHARED_INNER | \
-				TCR_IRGN_WBWA | \
-				TCR_ORGN_WBWA | \
-				TCR_T0SZ(ZYNQMO_VA_BITS)
-
-#define MEMORY_ATTR	PMD_SECT_AF | PMD_SECT_INNER_SHARE |	\
-			PMD_ATTRINDX(MT_NORMAL) |	\
-			PMD_TYPE_SECT
-#define DEVICE_ATTR	PMD_SECT_AF | PMD_SECT_PXN |	\
-			PMD_SECT_UXN | PMD_ATTRINDX(MT_DEVICE_NGNRNE) |	\
-			PMD_TYPE_SECT
-
-/* 4K size is required to place 512 entries in each level */
-#define TLB_TABLE_SIZE	0x1000
-
-struct attr_tbl {
-	u32 num;
-	u64 attr;
-};
-
-static struct attr_tbl attr_tbll1t0[4] = { {16, 0x0},
-					   {8, DEVICE_ATTR},
-					   {32, MEMORY_ATTR},
-					   {456, DEVICE_ATTR}
-					 };
-static struct attr_tbl attr_tbll2t3[4] = { {0x180, DEVICE_ATTR},
-					   {0x40, 0x0},
-					   {0x3F, DEVICE_ATTR},
-					   {0x1, MEMORY_ATTR}
-					 };
-
-/*
- * This mmu table looks as below
- * Level 0 table contains two entries to 512GB sizes. One is Level1 Table 0
- * and other Level1 Table1.
- * Level1 Table0 contains entries for each 1GB from 0 to 511GB.
- * Level1 Table1 contains entries for each 1GB from 512GB to 1TB.
- * Level2 Table0, Level2 Table1, Level2 Table2 and Level2 Table3 contains
- * entries for each 2MB starting from 0GB, 1GB, 2GB and 3GB respectively.
- */
-static void zynqmp_mmu_setup(void)
-{
-	int el;
-	u32 index_attr;
-	u64 i, section_l1t0, section_l1t1;
-	u64 section_l2t0, section_l2t1, section_l2t2, section_l2t3;
-	u64 *level0_table = (u64 *)gd->arch.tlb_addr;
-	u64 *level1_table_0 = (u64 *)(gd->arch.tlb_addr + TLB_TABLE_SIZE);
-	u64 *level1_table_1 = (u64 *)(gd->arch.tlb_addr + (2 * TLB_TABLE_SIZE));
-	u64 *level2_table_0 = (u64 *)(gd->arch.tlb_addr + (3 * TLB_TABLE_SIZE));
-	u64 *level2_table_1 = (u64 *)(gd->arch.tlb_addr + (4 * TLB_TABLE_SIZE));
-	u64 *level2_table_2 = (u64 *)(gd->arch.tlb_addr + (5 * TLB_TABLE_SIZE));
-	u64 *level2_table_3 = (u64 *)(gd->arch.tlb_addr + (6 * TLB_TABLE_SIZE));
-
-	level0_table[0] =
-		(u64)level1_table_0 | PMD_TYPE_TABLE;
-	level0_table[1] =
-		(u64)level1_table_1 | PMD_TYPE_TABLE;
-
-	/*
-	 * set level 1 table 0, covering 0 to 512GB
-	 * set level 1 table 1, covering 512GB to 1TB
-	 */
-	section_l1t0 = 0;
-	section_l1t1 = BLOCK_SIZE_L0;
-
-	index_attr = 0;
-	for (i = 0; i < 512; i++) {
-		level1_table_0[i] = section_l1t0;
-		level1_table_0[i] |= attr_tbll1t0[index_attr].attr;
-		attr_tbll1t0[index_attr].num--;
-		if (attr_tbll1t0[index_attr].num == 0)
-			index_attr++;
-		level1_table_1[i] = section_l1t1;
-		level1_table_1[i] |= DEVICE_ATTR;
-		section_l1t0 += BLOCK_SIZE_L1;
-		section_l1t1 += BLOCK_SIZE_L1;
-	}
-
-	level1_table_0[0] =
-		(u64)level2_table_0 | PMD_TYPE_TABLE;
-	level1_table_0[1] =
-		(u64)level2_table_1 | PMD_TYPE_TABLE;
-	level1_table_0[2] =
-		(u64)level2_table_2 | PMD_TYPE_TABLE;
-	level1_table_0[3] =
-		(u64)level2_table_3 | PMD_TYPE_TABLE;
-
-	section_l2t0 = 0;
-	section_l2t1 = section_l2t0 + BLOCK_SIZE_L1; /* 1GB */
-	section_l2t2 = section_l2t1 + BLOCK_SIZE_L1; /* 2GB */
-	section_l2t3 = section_l2t2 + BLOCK_SIZE_L1; /* 3GB */
-
-	index_attr = 0;
-
-	for (i = 0; i < 512; i++) {
-		level2_table_0[i] = section_l2t0 | MEMORY_ATTR;
-		level2_table_1[i] = section_l2t1 | MEMORY_ATTR;
-		level2_table_2[i] = section_l2t2 | DEVICE_ATTR;
-		level2_table_3[i] = section_l2t3 |
-				    attr_tbll2t3[index_attr].attr;
-		attr_tbll2t3[index_attr].num--;
-		if (attr_tbll2t3[index_attr].num == 0)
-			index_attr++;
-		section_l2t0 += BLOCK_SIZE_L2;
-		section_l2t1 += BLOCK_SIZE_L2;
-		section_l2t2 += BLOCK_SIZE_L2;
-		section_l2t3 += BLOCK_SIZE_L2;
-	}
-
-	/* flush new MMU table */
-	flush_dcache_range(gd->arch.tlb_addr,
-			   gd->arch.tlb_addr + gd->arch.tlb_size);
-
-	/* point TTBR to the new table */
-	el = current_el();
-	set_ttbr_tcr_mair(el, gd->arch.tlb_addr,
-			  ZYNQMP_TCR, MEMORY_ATTRIBUTES);
-
-	set_sctlr(get_sctlr() | CR_M);
-}
-
-int arch_cpu_init(void)
-{
-	icache_enable();
-	__asm_invalidate_dcache_all();
-	__asm_invalidate_tlb_all();
-	return 0;
-}
-
-/*
- * This function is called from lib/board.c.
- * It recreates MMU table in main memory. MMU and d-cache are enabled earlier.
- * There is no need to disable d-cache for this operation.
- */
-void enable_caches(void)
-{
-	/* The data cache is not active unless the mmu is enabled */
-	if (!(get_sctlr() & CR_M)) {
-		invalidate_dcache_all();
-		__asm_invalidate_tlb_all();
-		zynqmp_mmu_setup();
-	}
-	puts("Enabling Caches...\n");
-
-	set_sctlr(get_sctlr() | CR_C);
-}
-
-u64 *arch_get_page_table(void)
-{
-	return (u64 *)(gd->arch.tlb_addr + 0x3000);
-}
-#endif
include/configs/xilinx_zynqmp.h +2 -0
index da868b8..08f430c 100644
@@ -29,6 +29,8 @@
 #define CONFIG_SYS_MEMTEST_START	CONFIG_SYS_SDRAM_BASE
 #define CONFIG_SYS_MEMTEST_END		CONFIG_SYS_SDRAM_SIZE
 
+#define CONFIG_SYS_FULL_VA
+
 /* Have release address at the end of 256MB for now */
 #define CPU_RELEASE_ADDR	0xFFFFFF0
 
Signature · 1 line
1.8.5.6
#11 lore raw

PATCH v4 6/11 tegra: Replace home grown mmu code with generic table approach

In reply to Alexander Graf

Now that we have nice table driven page table creating code that gives us everything we need, move to that.

---

v1 -> v2:

- Move mmu tables into .c file

---
 arch/arm/mach-tegra/arm64-mmu.c   | 132 +++++---------------------------------
 include/configs/tegra210-common.h |   2 +
 2 files changed, 19 insertions(+), 115 deletions(-)
arch/arm/mach-tegra/arm64-mmu.c +17 -115
index c227652..501c4f0 100644
@@ -12,120 +12,22 @@
 #include <asm/system.h>
 #include <asm/armv8/mmu.h>
 
-DECLARE_GLOBAL_DATA_PTR;
-
-#define SECTION_SHIFT_L1	30UL
-#define SECTION_SHIFT_L2	21UL
-#define BLOCK_SIZE_L0		0x8000000000UL
-#define BLOCK_SIZE_L1		(1 << SECTION_SHIFT_L1)
-#define BLOCK_SIZE_L2		(1 << SECTION_SHIFT_L2)
-
-#define TCR_TG1_4K		(1 << 31)
-#define TCR_EPD1_DISABLE	(1 << 23)
-#define TEGRA_VA_BITS		40
-#define TEGRA_TCR		TCR_TG1_4K | \
-				TCR_EPD1_DISABLE | \
-				TCR_SHARED_OUTER | \
-				TCR_SHARED_INNER | \
-				TCR_IRGN_WBWA | \
-				TCR_ORGN_WBWA | \
-				TCR_T0SZ(TEGRA_VA_BITS)
-
-#define MEMORY_ATTR	PMD_SECT_AF | PMD_SECT_INNER_SHARE |	\
-			PMD_ATTRINDX(MT_NORMAL) |	\
-			PMD_TYPE_SECT
-#define DEVICE_ATTR	PMD_SECT_AF | PMD_SECT_PXN |	\
-			PMD_SECT_UXN | PMD_ATTRINDX(MT_DEVICE_NGNRNE) |	\
-			PMD_TYPE_SECT
-
-/* 4K size is required to place 512 entries in each level */
-#define TLB_TABLE_SIZE	0x1000
-
-/*
- * This mmu table looks as below
- * Level 0 table contains two entries to 512GB sizes. One is Level1 Table 0
- * and other Level1 Table1.
- * Level1 Table0 contains entries for each 1GB from 0 to 511GB.
- * Level1 Table1 contains entries for each 1GB from 512GB to 1TB.
- * Level2 Table0, Level2 Table1, Level2 Table2 and Level2 Table3 contains
- * entries for each 2MB starting from 0GB, 1GB, 2GB and 3GB respectively.
- */
-void mmu_setup(void)
-{
-	int el;
-	u64 i, section_l1t0, section_l1t1;
-	u64 section_l2t0, section_l2t1, section_l2t2, section_l2t3;
-	u64 *level0_table = (u64 *)gd->arch.tlb_addr;
-	u64 *level1_table_0 = (u64 *)(gd->arch.tlb_addr + TLB_TABLE_SIZE);
-	u64 *level1_table_1 = (u64 *)(gd->arch.tlb_addr + (2 * TLB_TABLE_SIZE));
-	u64 *level2_table_0 = (u64 *)(gd->arch.tlb_addr + (3 * TLB_TABLE_SIZE));
-	u64 *level2_table_1 = (u64 *)(gd->arch.tlb_addr + (4 * TLB_TABLE_SIZE));
-	u64 *level2_table_2 = (u64 *)(gd->arch.tlb_addr + (5 * TLB_TABLE_SIZE));
-	u64 *level2_table_3 = (u64 *)(gd->arch.tlb_addr + (6 * TLB_TABLE_SIZE));
-
-	/* Invalidate all table entries */
-	memset(level0_table, 0, PGTABLE_SIZE);
-
-	level0_table[0] =
-		(u64)level1_table_0 | PMD_TYPE_TABLE;
-	level0_table[1] =
-		(u64)level1_table_1 | PMD_TYPE_TABLE;
-
-	/*
-	 * set level 1 table 0, covering 0 to 512GB
-	 * set level 1 table 1, covering 512GB to 1TB
-	 */
-	section_l1t0 = 0;
-	section_l1t1 = BLOCK_SIZE_L0;
-
-	for (i = 0; i < 512; i++) {
-		level1_table_0[i] = section_l1t0;
-		if (i >= 4)
-			level1_table_0[i] |= MEMORY_ATTR;
-		level1_table_1[i] = section_l1t1;
-		level1_table_1[i] |= MEMORY_ATTR;
-		section_l1t0 += BLOCK_SIZE_L1;
-		section_l1t1 += BLOCK_SIZE_L1;
+static struct mm_region tegra_mem_map[] = {
+	{
+		.base = 0x0UL,
+		.size = 0x80000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
+			 PTE_BLOCK_NON_SHARE |
+			 PTE_BLOCK_PXN | PTE_BLOCK_UXN
+	}, {
+		.base = 0x80000000UL,
+		.size = 0xff80000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_NORMAL) |
+			 PTE_BLOCK_INNER_SHARE
+	}, {
+		/* List terminator */
+		0,
 	}
+};
 
-	level1_table_0[0] =
-		(u64)level2_table_0 | PMD_TYPE_TABLE;
-	level1_table_0[1] =
-		(u64)level2_table_1 | PMD_TYPE_TABLE;
-	level1_table_0[2] =
-		(u64)level2_table_2 | PMD_TYPE_TABLE;
-	level1_table_0[3] =
-		(u64)level2_table_3 | PMD_TYPE_TABLE;
-
-	section_l2t0 = 0;
-	section_l2t1 = section_l2t0 + BLOCK_SIZE_L1; /* 1GB */
-	section_l2t2 = section_l2t1 + BLOCK_SIZE_L1; /* 2GB */
-	section_l2t3 = section_l2t2 + BLOCK_SIZE_L1; /* 3GB */
-
-	for (i = 0; i < 512; i++) {
-		level2_table_0[i] = section_l2t0 | DEVICE_ATTR;
-		level2_table_1[i] = section_l2t1 | DEVICE_ATTR;
-		level2_table_2[i] = section_l2t2 | MEMORY_ATTR;
-		level2_table_3[i] = section_l2t3 | MEMORY_ATTR;
-		section_l2t0 += BLOCK_SIZE_L2;
-		section_l2t1 += BLOCK_SIZE_L2;
-		section_l2t2 += BLOCK_SIZE_L2;
-		section_l2t3 += BLOCK_SIZE_L2;
-	}
-
-	/* flush new MMU table */
-	flush_dcache_range(gd->arch.tlb_addr,
-			   gd->arch.tlb_addr + gd->arch.tlb_size);
-
-	/* point TTBR to the new table */
-	el = current_el();
-	set_ttbr_tcr_mair(el, gd->arch.tlb_addr,
-			  TEGRA_TCR, MEMORY_ATTRIBUTES);
-
-	set_sctlr(get_sctlr() | CR_M);
-}
-
-u64 *arch_get_page_table(void)
-{
-	return (u64 *)(gd->arch.tlb_addr + (3 * TLB_TABLE_SIZE));
-}
+struct mm_region *mem_map = tegra_mem_map;
include/configs/tegra210-common.h +2 -0
index 8f35a7b..2a6e317 100644
@@ -13,6 +13,8 @@
 /* Cortex-A57 uses a cache line size of 64 bytes */
 #define CONFIG_SYS_CACHELINE_SIZE	64
 
+#define CONFIG_SYS_FULL_VA
+
 /*
  * NS16550 Configuration
  */
Signature · 1 line
1.8.5.6
#12 lore raw

PATCH v4 7/11 vexpress64: Add MMU tables

In reply to Alexander Graf

There's no good excuse for running with caches disabled on AArch64, so let's just move the vexpress64 target to enable the MMU and run with caches on.

---

v1 -> v2:

- Move tables to .c file

---
 board/armltd/vexpress64/vexpress64.c | 21 +++++++++++++++++++++
 include/configs/vexpress_aemv8a.h    |  6 +++---
 2 files changed, 24 insertions(+), 3 deletions(-)
board/armltd/vexpress64/vexpress64.c +21 -0
index 6efc8c1..973b579 100644
@@ -14,6 +14,7 @@
 #include <dm/platdata.h>
 #include <dm/platform_data/serial_pl01x.h>
 #include "pcie.h"
+#include <asm/armv8/mmu.h>
 
 DECLARE_GLOBAL_DATA_PTR;
 
@@ -28,6 +29,26 @@ U_BOOT_DEVICE(vexpress_serials) = {
 	.platdata = &serial_platdata,
 };
 
+static struct mm_region vexpress64_mem_map[] = {
+	{
+		.base = 0x0UL,
+		.size = 0x80000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
+			 PTE_BLOCK_NON_SHARE |
+			 PTE_BLOCK_PXN | PTE_BLOCK_UXN
+	}, {
+		.base = 0x80000000UL,
+		.size = 0xff80000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_NORMAL) |
+			 PTE_BLOCK_INNER_SHARE
+	}, {
+		/* List terminator */
+		0,
+	}
+};
+
+struct mm_region *mem_map = vexpress64_mem_map;
+
 /* This function gets replaced by platforms supporting PCIe.
  * The replacement function, eg. on Juno, initialises the PCIe bus.
  */
include/configs/vexpress_aemv8a.h +3 -3
index 133041b..ddb9848 100644
@@ -19,9 +19,9 @@
 
 #define CONFIG_SUPPORT_RAW_INITRD
 
-/* Cache Definitions */
-#define CONFIG_SYS_DCACHE_OFF
-#define CONFIG_SYS_ICACHE_OFF
+/* MMU Definitions */
+#define CONFIG_SYS_CACHELINE_SIZE	64
+#define CONFIG_SYS_FULL_VA
 
 #define CONFIG_IDENT_STRING		" vexpress_aemv8a"
 #define CONFIG_BOOTP_VCI_STRING		"U-Boot.armv8.vexpress_aemv8a"
Signature · 1 line
1.8.5.6
#13 lore raw

PATCH v4 8/11 dwmmc: Increase retry timeout

In reply to Alexander Graf

When enable dcache on HiKey, we're running into MMC command timeouts because our retry loop is now faster than the eMMC (or an external SD card) can answer.

Increase the retry count to the same as the timeout value for status reports.

The real fix is obviously to not base this whole thing on a cycle counter but on real wall time, but that would be slightly more intrusive.

---
 drivers/mmc/dw_mmc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/mmc/dw_mmc.c +1 -1
index 909e3ca..7329f40 100644
@@ -189,7 +189,7 @@ static int dwmci_send_cmd(struct mmc *mmc, struct mmc_cmd *cmd,
 				 data ? DIV_ROUND_UP(data->blocks, 8) : 0);
 	int ret = 0, flags = 0, i;
 	unsigned int timeout = 100000;
-	u32 retry = 10000;
+	u32 retry = 100000;
 	u32 mask, ctrl;
 	ulong start = get_timer(0);
 	struct bounce_buffer bbstate;
Signature · 1 line
1.8.5.6
#14 lore raw

PATCH v4 9/11 hikey: Add MMU tables

In reply to Alexander Graf

The hikey runs with dcache disabled today. There really should be no reason not to use caches on AArch64, so let's add MMU definitions and enable the dcache.

---

v1 -> v2:

- Move tables to .c file

---
 board/hisilicon/hikey/hikey.c | 21 +++++++++++++++++++++
 include/configs/hikey.h       |  5 +++--
 2 files changed, 24 insertions(+), 2 deletions(-)
board/hisilicon/hikey/hikey.c +21 -0
index c4ae40b..1edc807 100644
@@ -19,6 +19,7 @@
 #include <asm/arch/periph.h>
 #include <asm/arch/pinmux.h>
 #include <asm/arch/hi6220.h>
+#include <asm/armv8/mmu.h>
 
 /*TODO drop this table in favour of device tree */
 static const struct hikey_gpio_platdata hi6220_gpio[] = {
@@ -87,6 +88,26 @@ U_BOOT_DEVICE(hikey_seriala) = {
 	.platdata = &serial_platdata,
 };
 
+static struct mm_region hikey_mem_map[] = {
+	{
+		.base = 0x0UL,
+		.size = 0x80000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_NORMAL) |
+			 PTE_BLOCK_INNER_SHARE
+	}, {
+		.base = 0x80000000UL,
+		.size = 0x80000000UL,
+		.attrs = PTE_BLOCK_MEMTYPE(MT_DEVICE_NGNRNE) |
+			 PTE_BLOCK_NON_SHARE |
+			 PTE_BLOCK_PXN | PTE_BLOCK_UXN
+	}, {
+		/* List terminator */
+		0,
+	}
+};
+
+struct mm_region *mem_map = hikey_mem_map;
+
 #ifdef CONFIG_BOARD_EARLY_INIT_F
 int board_uart_init(void)
 {
include/configs/hikey.h +3 -2
index 796861e..d33dcef 100644
@@ -21,8 +21,9 @@
 
 #define CONFIG_SUPPORT_RAW_INITRD
 
-/* Cache Definitions */
-#define CONFIG_SYS_DCACHE_OFF
+/* MMU Definitions */
+#define CONFIG_SYS_CACHELINE_SIZE	64
+#define CONFIG_SYS_FULL_VA
 
 #define CONFIG_IDENT_STRING		"hikey"
 
Signature · 1 line
1.8.5.6
#15 lore raw

PATCH v4 10/11 arm64: Remove non-full-va map code

In reply to Alexander Graf

By now the code to only have a single page table level with 64k page size and 42 bit address space is no longer used by any board in tree, so we can safely remove it.

To clean up code, move the layerscape mmu code to the new defines, removing redundant field definitions.

---

v1 -> v2:

- Add layerscape empty mmu table in .c file

---
 arch/arm/cpu/armv8/cache_v8.c                  | 90 ------------------------
 arch/arm/cpu/armv8/fsl-layerscape/cpu.c        | 37 ++++++++--
 arch/arm/include/asm/arch-fsl-layerscape/cpu.h | 94 +++++++++++++-------------
 arch/arm/include/asm/armv8/mmu.h               | 66 +-----------------
 arch/arm/include/asm/global_data.h             |  2 +-
 arch/arm/include/asm/system.h                  | 12 ++--
 doc/README.arm64                               | 20 ------
 include/configs/hikey.h                        |  1 -
 include/configs/tegra210-common.h              |  2 -
 include/configs/thunderx_88xx.h                | 14 ----
 include/configs/vexpress_aemv8a.h              |  1 -
 include/configs/xilinx_zynqmp.h                |  2 -
 12 files changed, 85 insertions(+), 256 deletions(-)
arch/arm/cpu/armv8/cache_v8.c +0 -90
index 55c6f2f..df15e00 100644
@@ -35,7 +35,6 @@ DECLARE_GLOBAL_DATA_PTR;
  *    off:          FFF
  */
 
-#ifdef CONFIG_SYS_FULL_VA
 static u64 get_tcr(int el, u64 *pips, u64 *pva_bits)
 {
 	u64 max_addr = 0;
@@ -386,38 +385,11 @@ static void setup_all_pgtables(void)
 	gd->arch.tlb_addr = tlb_addr;
 }
 
-#else
-
-inline void set_pgtable_section(u64 *page_table, u64 index, u64 section,
-			 u64 memory_type, u64 attribute)
-{
-	u64 value;
-
-	value = section | PMD_TYPE_SECT | PMD_SECT_AF;
-	value |= PMD_ATTRINDX(memory_type);
-	value |= attribute;
-	page_table[index] = value;
-}
-
-inline void set_pgtable_table(u64 *page_table, u64 index, u64 *table_addr)
-{
-	u64 value;
-
-	value = (u64)table_addr | PMD_TYPE_TABLE;
-	page_table[index] = value;
-}
-#endif
-
 /* to activate the MMU we need to set up virtual memory */
 __weak void mmu_setup(void)
 {
-#ifndef CONFIG_SYS_FULL_VA
-	bd_t *bd = gd->bd;
-	u64 *page_table = (u64 *)gd->arch.tlb_addr, i, j;
-#endif
 	int el;
 
-#ifdef CONFIG_SYS_FULL_VA
 	/* Set up page tables only once */
 	if (!gd->arch.tlb_fillptr)
 		setup_all_pgtables();
@@ -425,40 +397,6 @@ __weak void mmu_setup(void)
 	el = current_el();
 	set_ttbr_tcr_mair(el, gd->arch.tlb_addr, get_tcr(el, NULL, NULL),
 			  MEMORY_ATTRIBUTES);
-#else
-	/* Setup an identity-mapping for all spaces */
-	for (i = 0; i < (PGTABLE_SIZE >> 3); i++) {
-		set_pgtable_section(page_table, i, i << SECTION_SHIFT,
-				    MT_DEVICE_NGNRNE, PMD_SECT_NON_SHARE);
-	}
-
-	/* Setup an identity-mapping for all RAM space */
-	for (i = 0; i < CONFIG_NR_DRAM_BANKS; i++) {
-		ulong start = bd->bi_dram[i].start;
-		ulong end = bd->bi_dram[i].start + bd->bi_dram[i].size;
-		for (j = start >> SECTION_SHIFT;
-		     j < end >> SECTION_SHIFT; j++) {
-			set_pgtable_section(page_table, j, j << SECTION_SHIFT,
-					    MT_NORMAL, PMD_SECT_NON_SHARE);
-		}
-	}
-
-	/* load TTBR0 */
-	el = current_el();
-	if (el == 1) {
-		set_ttbr_tcr_mair(el, gd->arch.tlb_addr,
-				  TCR_EL1_RSVD | TCR_FLAGS | TCR_EL1_IPS_BITS,
-				  MEMORY_ATTRIBUTES);
-	} else if (el == 2) {
-		set_ttbr_tcr_mair(el, gd->arch.tlb_addr,
-				  TCR_EL2_RSVD | TCR_FLAGS | TCR_EL2_IPS_BITS,
-				  MEMORY_ATTRIBUTES);
-	} else {
-		set_ttbr_tcr_mair(el, gd->arch.tlb_addr,
-				  TCR_EL3_RSVD | TCR_FLAGS | TCR_EL3_IPS_BITS,
-				  MEMORY_ATTRIBUTES);
-	}
-#endif
 
 	/* enable the mmu */
 	set_sctlr(get_sctlr() | CR_M);
@@ -544,33 +482,6 @@ u64 *__weak arch_get_page_table(void) {
 	return NULL;
 }
 
-#ifndef CONFIG_SYS_FULL_VA
-void mmu_set_region_dcache_behaviour(phys_addr_t start, size_t size,
-				     enum dcache_option option)
-{
-	u64 *page_table = arch_get_page_table();
-	u64 upto, end;
-
-	if (page_table == NULL)
-		return;
-
-	end = ALIGN(start + size, (1 << MMU_SECTION_SHIFT)) >>
-	      MMU_SECTION_SHIFT;
-	start = start >> MMU_SECTION_SHIFT;
-	for (upto = start; upto < end; upto++) {
-		page_table[upto] &= ~PMD_ATTRINDX_MASK;
-		page_table[upto] |= PMD_ATTRINDX(option);
-	}
-	asm volatile("dsb sy");
-	__asm_invalidate_tlb_all();
-	asm volatile("dsb sy");
-	asm volatile("isb");
-	start = start << MMU_SECTION_SHIFT;
-	end = end << MMU_SECTION_SHIFT;
-	flush_dcache_range(start, end);
-	asm volatile("dsb sy");
-}
-#else
 static bool is_aligned(u64 addr, u64 size, u64 align)
 {
 	return !(addr & (align - 1)) && !(size & (align - 1));
@@ -652,7 +563,6 @@ void mmu_set_region_dcache_behaviour(phys_addr_t start, size_t size,
 	 */
 	flush_dcache_range(real_start, real_start + real_size);
 }
-#endif
 
 #else	/* CONFIG_SYS_DCACHE_OFF */
 
arch/arm/cpu/armv8/fsl-layerscape/cpu.c +32 -5
index 6ea28ed..7404bd9 100644
@@ -26,6 +26,14 @@
 
 DECLARE_GLOBAL_DATA_PTR;
 
+static struct mm_region layerscape_mem_map[] = {
+	{
+		/* List terminator */
+		0,
+	}
+};
+struct mm_region *mem_map = layerscape_mem_map;
+
 void cpu_name(char *name)
 {
 	struct ccsr_gur __iomem *gur = (void *)(CONFIG_SYS_FSL_GUTS_ADDR);
@@ -48,6 +56,25 @@ void cpu_name(char *name)
 }
 
 #ifndef CONFIG_SYS_DCACHE_OFF
+static void set_pgtable_section(u64 *page_table, u64 index, u64 section,
+			u64 memory_type, u64 attribute)
+{
+       u64 value;
+
+       value = section | PTE_TYPE_BLOCK | PTE_BLOCK_AF;
+       value |= PMD_ATTRINDX(memory_type);
+       value |= attribute;
+       page_table[index] = value;
+}
+
+static void set_pgtable_table(u64 *page_table, u64 index, u64 *table_addr)
+{
+       u64 value;
+
+       value = (u64)table_addr | PTE_TYPE_TABLE;
+       page_table[index] = value;
+}
+
 /*
  * Set the block entries according to the information of the table.
  */
@@ -114,10 +141,10 @@ static int find_table(const struct sys_mmu_table *list,
 
 		temp_base -= block_size;
 
-		if ((level_table[index - 1] & PMD_TYPE_MASK) ==
-		    PMD_TYPE_TABLE) {
+		if ((level_table[index - 1] & PTE_TYPE_MASK) ==
+		    PTE_TYPE_TABLE) {
 			level_table = (u64 *)(level_table[index - 1] &
-				      ~PMD_TYPE_MASK);
+				      ~PTE_TYPE_MASK);
 			level++;
 			continue;
 		} else {
@@ -220,7 +247,7 @@ static inline int final_secure_ddr(u64 *level0_table,
 	struct table_info table = {};
 	struct sys_mmu_table ddr_entry = {
 		0, 0, BLOCK_SIZE_L1, MT_NORMAL,
-		PMD_SECT_OUTER_SHARE | PMD_SECT_NS
+		PTE_BLOCK_OUTER_SHARE | PTE_BLOCK_NS
 	};
 	u64 index;
 
@@ -243,7 +270,7 @@ static inline int final_secure_ddr(u64 *level0_table,
 	ddr_entry.virt_addr = phys_addr;
 	ddr_entry.phys_addr = phys_addr;
 	ddr_entry.size = CONFIG_SYS_MEM_RESERVE_SECURE;
-	ddr_entry.attribute = PMD_SECT_OUTER_SHARE;
+	ddr_entry.attribute = PTE_BLOCK_OUTER_SHARE;
 	ret = find_table(&ddr_entry, &table, level0_table);
 	if (ret) {
 		printf("MMU error: could not find secure ddr table\n");
arch/arm/include/asm/arch-fsl-layerscape/cpu.h +47 -47

191 lines not on this page · raw message

arch/arm/include/asm/armv8/mmu.h +1 -65

113 lines not on this page · raw message

arch/arm/include/asm/global_data.h +1 -1
index 259daa1..77d2653 100644
@@ -39,7 +39,7 @@ struct arch_global_data {
 #if !(defined(CONFIG_SYS_ICACHE_OFF) && defined(CONFIG_SYS_DCACHE_OFF))
 	unsigned long tlb_addr;
 	unsigned long tlb_size;
-#if defined(CONFIG_SYS_FULL_VA)
+#if defined(CONFIG_ARM64)
 	unsigned long tlb_fillptr;
 	unsigned long tlb_emerg;
 #endif
arch/arm/include/asm/system.h +4 -8
index 9b1cbf2..ac1173d 100644
@@ -17,18 +17,14 @@
 #define CR_WXN		(1 << 19)	/* Write Permision Imply XN	*/
 #define CR_EE		(1 << 25)	/* Exception (Big) Endian	*/
 
-/* 2MB granularity */
-#define MMU_SECTION_SHIFT	21
-#define MMU_SECTION_SIZE	(1 << MMU_SECTION_SHIFT)
-
 #ifndef __ASSEMBLY__
 
-#ifndef CONFIG_SYS_FULL_VA
-#define PGTABLE_SIZE	(0x10000)
-#else
 u64 get_page_table_size(void);
 #define PGTABLE_SIZE	get_page_table_size()
-#endif
+
+/* 2MB granularity */
+#define MMU_SECTION_SHIFT	21
+#define MMU_SECTION_SIZE	(1 << MMU_SECTION_SHIFT)
 
 enum dcache_option {
 	DCACHE_OFF = 0x3,
doc/README.arm64 +0 -20
index de669cb..f658fa2 100644
@@ -36,26 +36,6 @@ Notes
 6. CONFIG_ARM64 instead of CONFIG_ARMV8 is used to distinguish aarch64 and
    aarch32 specific codes.
 
-7. CONFIG_SYS_FULL_VA is used to enable 2-level page tables. For cores
-   supporting 64k pages it allows usage of full 48+ virtual/physical addresses
-
-   Enabling this option requires the following ones to be defined:
-       - CONFIG_SYS_MEM_MAP - an array of 'struct mm_region' describing the
-         system memory map (start, length, attributes)
-       - CONFIG_SYS_MEM_MAP_SIZE - number of entries in CONFIG_SYS_MEM_MAP
-       - CONFIG_SYS_PTL1_ENTRIES - number of 1st level page table entries
-       - CONFIG_SYS_PTL2_ENTRIES - number of 1nd level page table entries
-         for the largest CONFIG_SYS_MEM_MAP entry
-       - CONFIG_COREID_MASK - the mask value used to get the core from the
-         MPIDR_EL1 register
-       - CONFIG_SYS_PTL2_BITS - number of bits addressed by the 2nd level
-         page tables
-       - CONFIG_SYS_BLOCK_SHIFT - number of bits addressed by a single block
-         entry from L2 page tables
-       - CONFIG_SYS_PGTABLE_SIZE - total size of the page table
-       - CONFIG_SYS_TCR_EL{1,2,3}_IPS_BITS - the IPS field of the TCR_EL{1,2,3}
-
-
 
 
 Contributor
include/configs/hikey.h +0 -1
index d33dcef..2d9ace9 100644
@@ -23,7 +23,6 @@
 
 /* MMU Definitions */
 #define CONFIG_SYS_CACHELINE_SIZE	64
-#define CONFIG_SYS_FULL_VA
 
 #define CONFIG_IDENT_STRING		"hikey"
 
include/configs/tegra210-common.h +0 -2
index 2a6e317..8f35a7b 100644
@@ -13,8 +13,6 @@
 /* Cortex-A57 uses a cache line size of 64 bytes */
 #define CONFIG_SYS_CACHELINE_SIZE	64
 
-#define CONFIG_SYS_FULL_VA
-
 /*
  * NS16550 Configuration
  */
include/configs/thunderx_88xx.h +0 -14
index 64e4616..736d0a5 100644
@@ -22,22 +22,8 @@
 
 #define MEM_BASE			0x00500000
 
-#define CONFIG_SYS_FULL_VA
-
 #define CONFIG_SYS_LOWMEM_BASE		MEM_BASE
 
-#define CONFIG_SYS_MEM_MAP_SIZE		3
-
-#define CONFIG_SYS_VA_BITS		48
-#define CONFIG_SYS_PTL2_BITS		42
-#define CONFIG_SYS_BLOCK_SHIFT		29
-#define CONFIG_SYS_PTL1_ENTRIES		64
-#define CONFIG_SYS_PTL2_ENTRIES		8192
-
-#define CONFIG_SYS_PGTABLE_SIZE		\
-	((CONFIG_SYS_PTL1_ENTRIES + \
-	  CONFIG_SYS_MEM_MAP_SIZE * CONFIG_SYS_PTL2_ENTRIES) * 8)
-
 /* Link Definitions */
 #define CONFIG_SYS_TEXT_BASE		0x00500000
 #define CONFIG_SYS_INIT_SP_ADDR		(CONFIG_SYS_SDRAM_BASE + 0x7fff0)
include/configs/vexpress_aemv8a.h +0 -1
index ddb9848..2949170 100644
@@ -21,7 +21,6 @@
 
 /* MMU Definitions */
 #define CONFIG_SYS_CACHELINE_SIZE	64
-#define CONFIG_SYS_FULL_VA
 
 #define CONFIG_IDENT_STRING		" vexpress_aemv8a"
 #define CONFIG_BOOTP_VCI_STRING		"U-Boot.armv8.vexpress_aemv8a"
include/configs/xilinx_zynqmp.h +0 -2
index 08f430c..da868b8 100644
@@ -29,8 +29,6 @@
 #define CONFIG_SYS_MEMTEST_START	CONFIG_SYS_SDRAM_BASE
 #define CONFIG_SYS_MEMTEST_END		CONFIG_SYS_SDRAM_SIZE
 
-#define CONFIG_SYS_FULL_VA
-
 /* Have release address@the end of 256MB for now */
 #define CPU_RELEASE_ADDR	0xFFFFFF0
 
Signature · 1 line
1.8.5.6
#16 lore raw

PATCH v4 11/11 arm64: Only allow dcache disabled in SPL builds

In reply to Alexander Graf

Now that we have an easy way to describe memory regions and enable the MMU, there really shouldn't be anything holding people back from running with caches enabled on AArch64. To make sure people catch early if they're missing on the caching fun, give them a compile error.

---
 arch/arm/cpu/armv8/cache_v8.c | 9 +++++++++
 1 file changed, 9 insertions(+)
arch/arm/cpu/armv8/cache_v8.c +9 -0
index df15e00..d1bd06b 100644
@@ -566,6 +566,15 @@ void mmu_set_region_dcache_behaviour(phys_addr_t start, size_t size,
 
 #else	/* CONFIG_SYS_DCACHE_OFF */
 
+/*
+ * For SPL builds, we may want to not have dcache enabled. Any real U-Boot
+ * running however really wants to have dcache and the MMU active. Check that
+ * everything is sane and give the developer a hint if it isn't.
+ */
+#ifndef CONFIG_SPL_BUILD
+#error Please describe your MMU layout in CONFIG_SYS_MEM_MAP and enable dcache.
+#endif
+
 void invalidate_dcache_all(void)
 {
 }
Signature · 1 line
1.8.5.6
#17 lore raw

v4 cover arm64: Unify MMU code v3

In reply to Alexander Graf
Quoted 22 lines · Alexander Graf

Howdy,

Currently on arm64 there is a big pile of mess when it comes to MMU support and page tables. Each board does its own little thing and the generic code is pretty dumb and nobody actually uses it.

This patch set tries to clean that up. After this series is applied, all boards except for the FSL Layerscape ones are converted to the new generic page table logic and have icache+dcache enabled.

The new code always uses 4k page size. It dynamically allocates 1G or 2M pages for ranges that fit. When a dcache attribute request comes in that requires a smaller granularity than our previous allocation could fulfill, pages get automatically split.

I have tested and verified the code works on HiKey (bare metal), vexpress64 (Foundation Model) and zynqmp (QEMU). The TX1 target is untested, but given the simplicity of the maps I doubt it'll break. ThunderX in theory should also work, but I haven't tested it. I would be very happy if people with access to those system could give the patch set a try.

Quoted 7 lines

With this we're a big step closer to a good base line for EFI payload support, since we can now just require that all boards always have dcache enabled.

I would also be incredibly happy if some Freescale people could look at their MMU code and try to unify it into the now cleaned up generic code. I don't think we're far off here.

(On p2371-0000 and p2371-2180)