Skip to contentSkip to conversation

arm64: mm: Model LVA support as a CPU feature

PATCH In review Ard Biesheuvel 4 participants 15 messages Nov 15, 2022 – Dec 1, 2022 arm64

lore
Share & tools
lore
#1 lore raw

PATCH 1/3 arm64: mm: get rid of kimage_vaddr global variable

In reply to Ard Biesheuvel

We store the address of _text in kimage_vaddr, but since commit 09e3c22a86f6889d ("arm64: Use a variable to store non-global mappings decision"), we no longer reference this variable from modules so we no longer need to export it.

In fact, we don't need it at all so let's just get rid of it.

---
 arch/arm64/include/asm/memory.h | 6 ++----
 arch/arm64/kernel/head.S        | 2 +-
 arch/arm64/mm/mmu.c             | 3 ---
 3 files changed, 3 insertions(+), 8 deletions(-)
arch/arm64/include/asm/memory.h +2 -4
index 78e5163836a0ab95..a4e1d832a15a2d7a 100644
@@ -182,6 +182,7 @@
 #include <linux/types.h>
 #include <asm/boot.h>
 #include <asm/bug.h>
+#include <asm/sections.h>
 
 #if VA_BITS > 48
 extern u64			vabits_actual;
@@ -193,15 +194,12 @@ extern s64			memstart_addr;
 /* PHYS_OFFSET - the physical address of the start of memory. */
 #define PHYS_OFFSET		({ VM_BUG_ON(memstart_addr & 1); memstart_addr; })
 
-/* the virtual base of the kernel image */
-extern u64			kimage_vaddr;
-
 /* the offset between the kernel virtual and physical mappings */
 extern u64			kimage_voffset;
 
 static inline unsigned long kaslr_offset(void)
 {
-	return kimage_vaddr - KIMAGE_VADDR;
+	return (u64)&_text - KIMAGE_VADDR;
 }
 
 static inline bool kaslr_enabled(void)
arch/arm64/kernel/head.S +1 -1
index 3ea5bf0a6e177e51..3b3c5e8e84af890e 100644
@@ -399,7 +399,7 @@ SYM_FUNC_START_LOCAL(__primary_switched)
 
 	str_l	x21, __fdt_pointer, x5		// Save FDT pointer
 
-	ldr_l	x4, kimage_vaddr		// Save the offset between
+	adrp	x4, _text			// Save the offset between
 	sub	x4, x4, x0			// the kernel virtual and
 	str_l	x4, kimage_voffset, x5		// physical mappings
 
arch/arm64/mm/mmu.c +0 -3
index 6942255056aed5ae..a9714b00f5410d7d 100644
@@ -50,9 +50,6 @@ u64 vabits_actual __ro_after_init = VA_BITS_MIN;
 EXPORT_SYMBOL(vabits_actual);
 #endif
 
-u64 kimage_vaddr __ro_after_init = (u64)&_text;
-EXPORT_SYMBOL(kimage_vaddr);
-
 u64 kimage_voffset __ro_after_init;
 EXPORT_SYMBOL(kimage_voffset);
 
Signature · 7 lines
2.35.1


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#0 lore raw

cover arm64: mm: Model LVA support as a CPU feature

Treat LVA support (52-bit virtual addressing) as a CPU feature so we can rely on code patching, reporting and overrides as usual.

Currently, we enable LVA really early, and have special treatment of it in the boot paths. However, given that both the ID map and the TTBR1 mapping of the kernel image are guarenteed to require 48 bits at most, we can boot with 48-bit virtual addressing initially, and switch to 52-bit addressing once we are running from C code and have parsed the command line to look for CPU feature overrides.

Patch #1 is a drive-by cleanup that can be applied independently.

Patch #2 creates the LVA CPU feature and wires it up.

Patch #3 introduces the MMFR2 CPU feature override, as well as a arm64.nolva command line option to force 48-bit virtual addressing even if the CPU supports LVA. While at it, E0PD is wired up as an override as well, given that it sits in MMFR2 as well, and that its value is taken into account very early too.

Note that this applies onto my series that further refactors the boot code [0], after a previous refactor landed in v6.0.

[0] https://lore.kernel.org/all/[email protected]/

Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Anshuman Khandual <[email protected]>
Cc Joey Gouly <[email protected]>

Ard Biesheuvel (3):
arm64: mm: get rid of kimage_vaddr global variable
arm64: mm: Handle LVA support as a CPU feature
arm64: mm: Add feature override support for LVA and E0PD

 arch/arm64/include/asm/assembler.h    | 17 +++++++------
 arch/arm64/include/asm/cpufeature.h   |  1 +
 arch/arm64/include/asm/memory.h       | 10 ++++----
 arch/arm64/kernel/cpufeature.c        | 19 +++++++++++++-
 arch/arm64/kernel/head.S              | 26 +++-----------------
 arch/arm64/kernel/image-vars.h        |  1 +
 arch/arm64/kernel/pi/idreg-override.c |  8 +++++-
 arch/arm64/kernel/pi/map_kernel.c     | 16 ++++++++++++
 arch/arm64/kernel/sleep.S             |  3 ---
 arch/arm64/mm/mmu.c                   |  8 ------
 arch/arm64/mm/proc.S                  | 16 ++++++------
 arch/arm64/tools/cpucaps              |  1 +
 12 files changed, 70 insertions(+), 56 deletions(-)
Signature · 7 lines
2.35.1


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#12 lore raw

PATCH 3/3 arm64: mm: Add feature override support for LVA and E0PD

Add support for overriding the VARange and E0PD fields of the MMFR2 CPU ID register. This permits the associated features to be overridden early enough for the boot code that creates the kernel mapping to take it into account.

---
 arch/arm64/include/asm/assembler.h    | 17 ++++++++++-------
 arch/arm64/include/asm/cpufeature.h   |  1 +
 arch/arm64/kernel/cpufeature.c        |  6 +++++-
 arch/arm64/kernel/image-vars.h        |  1 +
 arch/arm64/kernel/pi/idreg-override.c |  8 +++++++-
 arch/arm64/kernel/pi/map_kernel.c     |  4 ++++
 6 files changed, 28 insertions(+), 9 deletions(-)
arch/arm64/include/asm/assembler.h +10 -7
index e5957a53be3983ac..941082cfb788151a 100644
@@ -604,18 +604,21 @@ alternative_endif
 	.endm
 
 /*
- * Offset ttbr1 to allow for 48-bit kernel VAs set with 52-bit PTRS_PER_PGD.
+ * If the kernel is built for 52-bit virtual addressing but the hardware only
+ * supports 48 bits, we cannot program the pgdir address into TTBR1 directly,
+ * but we have to add an offset so that the TTBR1 address corresponds with the
+ * pgdir entry that covers the lowest 48-bit addressable VA.
+ *
  * orr is used as it can cover the immediate value (and is idempotent).
- * In future this may be nop'ed out when dealing with 52-bit kernel VAs.
  * 	ttbr: Value of ttbr to set, modified.
  */
 	.macro	offset_ttbr1, ttbr, tmp
 #ifdef CONFIG_ARM64_VA_BITS_52
-	mrs_s	\tmp, SYS_ID_AA64MMFR2_EL1
-	and	\tmp, \tmp, #(0xf << ID_AA64MMFR2_EL1_VARange_SHIFT)
-	cbnz	\tmp, .Lskipoffs_\@
-	orr	\ttbr, \ttbr, #TTBR1_BADDR_4852_OFFSET
-.Lskipoffs_\@ :
+	mrs	\tmp, tcr_el1
+	and	\tmp, \tmp, #TCR_T1SZ_MASK
+	cmp	\tmp, #TCR_T1SZ(VA_BITS_MIN)
+	orr	\tmp, \ttbr, #TTBR1_BADDR_4852_OFFSET
+	csel	\ttbr, \tmp, \ttbr, eq
 #endif
 	.endm
 
arch/arm64/include/asm/cpufeature.h +1 -0
index 4b5c639a5a0a7fab..7aa9cd4fc67f7c61 100644
@@ -911,6 +911,7 @@ static inline unsigned int get_vmid_bits(u64 mmfr1)
 struct arm64_ftr_reg *get_arm64_ftr_reg(u32 sys_id);
 
 extern struct arm64_ftr_override id_aa64mmfr1_override;
+extern struct arm64_ftr_override id_aa64mmfr2_override;
 extern struct arm64_ftr_override id_aa64pfr0_override;
 extern struct arm64_ftr_override id_aa64pfr1_override;
 extern struct arm64_ftr_override id_aa64zfr0_override;
arch/arm64/kernel/cpufeature.c +5 -1
index b44aece5024c3e2d..469d8b31487e88b6 100644
@@ -637,6 +637,7 @@ static const struct arm64_ftr_bits ftr_raz[] = {
 	__ARM64_FTR_REG_OVERRIDE(#id, id, table, &no_override)
 
 struct arm64_ftr_override id_aa64mmfr1_override;
+struct arm64_ftr_override id_aa64mmfr2_override;
 struct arm64_ftr_override id_aa64pfr0_override;
 struct arm64_ftr_override id_aa64pfr1_override;
 struct arm64_ftr_override id_aa64zfr0_override;
@@ -703,7 +704,8 @@ static const struct __ftr_reg_entry {
 	ARM64_FTR_REG(SYS_ID_AA64MMFR0_EL1, ftr_id_aa64mmfr0),
 	ARM64_FTR_REG_OVERRIDE(SYS_ID_AA64MMFR1_EL1, ftr_id_aa64mmfr1,
 			       &id_aa64mmfr1_override),
-	ARM64_FTR_REG(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2),
+	ARM64_FTR_REG_OVERRIDE(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2,
+			       &id_aa64mmfr2_override),
 
 	/* Op1 = 0, CRn = 1, CRm = 2 */
 	ARM64_FTR_REG(SYS_ZCR_EL1, ftr_zcr),
@@ -1605,6 +1607,8 @@ bool kaslr_requires_kpti(void)
 	 */
 	if (IS_ENABLED(CONFIG_ARM64_E0PD)) {
 		u64 mmfr2 = read_sysreg_s(SYS_ID_AA64MMFR2_EL1);
+		mmfr2 &= ~id_aa64mmfr2_override.mask;
+		mmfr2 |= id_aa64mmfr2_override.val;
 		if (cpuid_feature_extract_unsigned_field(mmfr2,
 						ID_AA64MMFR2_EL1_E0PD_SHIFT))
 			return false;
arch/arm64/kernel/image-vars.h +1 -0
index 5bd878f414d85366..6626f95f7ead0682 100644
@@ -46,6 +46,7 @@ PROVIDE(__pi_memstart_offset_seed	= memstart_offset_seed);
 PROVIDE(__pi_id_aa64isar1_override	= id_aa64isar1_override);
 PROVIDE(__pi_id_aa64isar2_override	= id_aa64isar2_override);
 PROVIDE(__pi_id_aa64mmfr1_override	= id_aa64mmfr1_override);
+PROVIDE(__pi_id_aa64mmfr2_override	= id_aa64mmfr2_override);
 PROVIDE(__pi_id_aa64pfr0_override	= id_aa64pfr0_override);
 PROVIDE(__pi_id_aa64pfr1_override	= id_aa64pfr1_override);
 PROVIDE(__pi_id_aa64smfr0_override	= id_aa64smfr0_override);
arch/arm64/kernel/pi/idreg-override.c +7 -1
index 662c3d21e150e7f9..3be2f887e6cae29f 100644
@@ -139,12 +139,17 @@ DEFINE_OVERRIDE(6, sw_features, "arm64_sw", arm64_sw_feature_override,
 		FIELD("nowxn", ARM64_SW_FEATURE_OVERRIDE_NOWXN),
 		{});
 
+DEFINE_OVERRIDE(7, mmfr2, "id_aa64mmfr2", id_aa64mmfr2_override,
+		FIELD("varange", ID_AA64MMFR2_EL1_VARange_SHIFT),
+		FIELD("e0pd", ID_AA64MMFR2_EL1_E0PD_SHIFT),
+		{});
+
 /*
  * regs[] is populated by R_AARCH64_PREL32 directives invisible to the compiler
  * so it cannot be static or const, or the compiler might try to use constant
  * propagation on the values.
  */
-asmlinkage s32 regs[7] __initdata = { [0 ... ARRAY_SIZE(regs) - 1] = S32_MAX };
+asmlinkage s32 regs[8] __initdata = { [0 ... ARRAY_SIZE(regs) - 1] = S32_MAX };
 
 static struct arm64_ftr_override * __init reg_override(int i)
 {
@@ -170,6 +175,7 @@ static const struct {
 	{ "nokaslr",			"arm64_sw.nokaslr=1" },
 	{ "rodata=off",			"arm64_sw.rodataoff=1 arm64_sw.nowxn=1" },
 	{ "arm64.nowxn",		"arm64_sw.nowxn=1" },
+	{ "arm64.nolva",		"id_aa64mmfr2.varange=0" },
 };
 
 static int __init find_field(const char *cmdline, char *opt, int len,
arch/arm64/kernel/pi/map_kernel.c +4 -0
index 3504e3266b02f636..c3edd207e3c031a2 100644
@@ -118,6 +118,8 @@ static bool __init arm64_early_this_cpu_has_e0pd(void)
 		return false;
 
 	mmfr2 = read_sysreg_s(SYS_ID_AA64MMFR2_EL1);
+	mmfr2 &= ~id_aa64mmfr2_override.mask;
+	mmfr2 |= id_aa64mmfr2_override.val;
 	return cpuid_feature_extract_unsigned_field(mmfr2,
 						    ID_AA64MMFR2_EL1_E0PD_SHIFT);
 }
@@ -127,6 +129,8 @@ static bool __init arm64_early_this_cpu_has_lva(void)
 	u64 mmfr2;
 
 	mmfr2 = read_sysreg_s(SYS_ID_AA64MMFR2_EL1);
+	mmfr2 &= ~id_aa64mmfr2_override.mask;
+	mmfr2 |= id_aa64mmfr2_override.val;
 	return cpuid_feature_extract_unsigned_field(mmfr2,
 						    ID_AA64MMFR2_EL1_VARange_SHIFT);
 }
Signature · 7 lines
2.35.1


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#3 lore raw

PATCH 2/3 arm64: mm: Handle LVA support as a CPU feature

In reply to Ard Biesheuvel

Currently, we detect CPU support for 52-bit virtual addressing (LVA) extremely early, before creating the kernel page tables or enabling the MMU. We cannot override the feature this early, and so large virtual addressing is always enabled on CPUs that implement support for it if the software support for it was enabled at build time. It also means we rely on non-trivial code in asm to deal with this feature.

Given that both the ID map and the TTBR1 mapping of the kernel image are guaranteed to be 48-bit addressable, it is not actually necessary to enable support this early, and instead, we can model it as a CPU feature. That way, we can rely on code patching to get the correct TCR.T1SZ values programmed on secondary boot and suspend from resume.

On the primary boot path, we simply enable the MMU with 48-bit virtual addressing initially, and update TCR.T1SZ from C code if LVA is supported, right before creating the kernel mapping. Given that TTBR1 still points to reserved_pg_dir at this point, updating TCR.T1SZ should be safe without the need for explicit TLB maintenance.

Since this gets rid of all accesses to the vabits_actual variable from asm code that occurred before TCR.T1SZ had been programmed, we no longer have a need for this variable, and we can replace it with a C expression that produces the correct value directly, based on the value of TCR.T1SZ.

---
 arch/arm64/include/asm/memory.h   |  4 +++-
 arch/arm64/kernel/cpufeature.c    | 13 +++++++++++
 arch/arm64/kernel/head.S          | 24 +++-----------------
 arch/arm64/kernel/pi/map_kernel.c | 12 ++++++++++
 arch/arm64/kernel/sleep.S         |  3 ---
 arch/arm64/mm/mmu.c               |  5 ----
 arch/arm64/mm/proc.S              | 16 ++++++-------
 arch/arm64/tools/cpucaps          |  1 +
 8 files changed, 39 insertions(+), 39 deletions(-)
arch/arm64/include/asm/memory.h +3 -1
index a4e1d832a15a2d7a..20e15c3f4589bd38 100644
@@ -183,9 +183,11 @@
 #include <asm/boot.h>
 #include <asm/bug.h>
 #include <asm/sections.h>
+#include <asm/sysreg.h>
 
 #if VA_BITS > 48
-extern u64			vabits_actual;
+// For reasons of #include hell, we can't use TCR_T1SZ_OFFSET/TCR_T1SZ_MASK here
+#define vabits_actual		(64 - ((read_sysreg(tcr_el1) >> 16) & 63))
 #else
 #define vabits_actual		((u64)VA_BITS)
 #endif
arch/arm64/kernel/cpufeature.c +13 -0
index eca9df123a8b354b..b44aece5024c3e2d 100644
@@ -2654,6 +2654,19 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
 		.matches = has_cpuid_feature,
 		.cpu_enable = cpu_trap_el0_impdef,
 	},
+#ifdef CONFIG_ARM64_VA_BITS_52
+	{
+		.desc = "52-bit Virtual Addressing (LVA)",
+		.capability = ARM64_HAS_LVA,
+		.type = ARM64_CPUCAP_BOOT_CPU_FEATURE,
+		.sys_reg = SYS_ID_AA64MMFR2_EL1,
+		.sign = FTR_UNSIGNED,
+		.field_width = 4,
+		.field_pos = ID_AA64MMFR2_EL1_VARange_SHIFT,
+		.matches = has_cpuid_feature,
+		.min_field_value = ID_AA64MMFR2_EL1_VARange_52,
+	},
+#endif
 	{},
 };
 
arch/arm64/kernel/head.S +3 -21
index 3b3c5e8e84af890e..6abf513189c7ebc9 100644
@@ -80,7 +80,6 @@
 	 *  x20        primary_entry() .. __primary_switch()    CPU boot mode
 	 *  x21        primary_entry() .. start_kernel()        FDT pointer passed at boot in x0
 	 *  x22        create_idmap() .. start_kernel()         ID map VA of the DT blob
-	 *  x25        primary_entry() .. start_kernel()        supported VA size
 	 *  x28        create_idmap()                           callee preserved temp register
 	 */
 SYM_CODE_START(primary_entry)
@@ -95,14 +94,6 @@ SYM_CODE_START(primary_entry)
 	 * On return, the CPU will be ready for the MMU to be turned on and
 	 * the TCR will have been set.
 	 */
-#if VA_BITS > 48
-	mrs_s	x0, SYS_ID_AA64MMFR2_EL1
-	tst	x0, #0xf << ID_AA64MMFR2_EL1_VARange_SHIFT
-	mov	x0, #VA_BITS
-	mov	x25, #VA_BITS_MIN
-	csel	x25, x25, x0, eq
-	mov	x0, x25
-#endif
 	bl	__cpu_setup			// initialise processor
 	b	__primary_switch
 SYM_CODE_END(primary_entry)
@@ -406,11 +397,6 @@ SYM_FUNC_START_LOCAL(__primary_switched)
 	mov	x0, x20
 	bl	set_cpu_boot_mode_flag
 
-#if VA_BITS > 48
-	adr_l	x8, vabits_actual		// Set this early so KASAN early init
-	str	x25, [x8]			// ... observes the correct value
-	dc	civac, x8			// Make visible to booting secondaries
-#endif
 #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
 	bl	kasan_early_init
 #endif
@@ -525,9 +511,6 @@ SYM_FUNC_START_LOCAL(secondary_startup)
 	mov	x20, x0				// preserve boot mode
 	bl	finalise_el2
 	bl	__cpu_secondary_check52bitva
-#if VA_BITS > 48
-	ldr_l	x0, vabits_actual
-#endif
 	bl	__cpu_setup			// initialise processor
 	adrp	x1, swapper_pg_dir
 	adrp	x2, idmap_pg_dir
@@ -628,10 +611,9 @@ SYM_FUNC_END(__enable_mmu)
 
 SYM_FUNC_START(__cpu_secondary_check52bitva)
 #if VA_BITS > 48
-	ldr_l	x0, vabits_actual
-	cmp	x0, #52
-	b.ne	2f
-
+alternative_if_not ARM64_HAS_LVA
+	ret
+alternative_else_nop_endif
 	mrs_s	x0, SYS_ID_AA64MMFR2_EL1
 	and	x0, x0, #(0xf << ID_AA64MMFR2_EL1_VARange_SHIFT)
 	cbnz	x0, 2f
arch/arm64/kernel/pi/map_kernel.c +12 -0
index 2bbf017147830bbe..3504e3266b02f636 100644
@@ -122,6 +122,15 @@ static bool __init arm64_early_this_cpu_has_e0pd(void)
 						    ID_AA64MMFR2_EL1_E0PD_SHIFT);
 }
 
+static bool __init arm64_early_this_cpu_has_lva(void)
+{
+	u64 mmfr2;
+
+	mmfr2 = read_sysreg_s(SYS_ID_AA64MMFR2_EL1);
+	return cpuid_feature_extract_unsigned_field(mmfr2,
+						    ID_AA64MMFR2_EL1_VARange_SHIFT);
+}
+
 static bool __init arm64_early_this_cpu_has_pac(void)
 {
 	u64 isar1, isar2;
@@ -274,6 +283,9 @@ asmlinkage void __init early_map_kernel(u64 boot_status, void *fdt)
 	/* Parse the command line for CPU feature overrides */
 	init_feature_override(boot_status, fdt, chosen);
 
+	if (VA_BITS > VA_BITS_MIN && arm64_early_this_cpu_has_lva())
+		sysreg_clear_set(tcr_el1, TCR_T1SZ_MASK, TCR_T1SZ(VA_BITS));
+
 	if (IS_ENABLED(CONFIG_ARM64_WXN) &&
 	    cpuid_feature_extract_unsigned_field(arm64_sw_feature_override.val,
 						 ARM64_SW_FEATURE_OVERRIDE_NOWXN))
arch/arm64/kernel/sleep.S +0 -3
index 97c9de57725dfddb..617f78ad43a185c2 100644
@@ -101,9 +101,6 @@ SYM_FUNC_END(__cpu_suspend_enter)
 SYM_CODE_START(cpu_resume)
 	bl	init_kernel_el
 	bl	finalise_el2
-#if VA_BITS > 48
-	ldr_l	x0, vabits_actual
-#endif
 	bl	__cpu_setup
 	/* enable the MMU early - so we can access sleep_save_stash by va */
 	adrp	x1, swapper_pg_dir
arch/arm64/mm/mmu.c +0 -5
index a9714b00f5410d7d..63fb62e16a1f8873 100644
@@ -45,11 +45,6 @@
 
 int idmap_t0sz __ro_after_init;
 
-#if VA_BITS > 48
-u64 vabits_actual __ro_after_init = VA_BITS_MIN;
-EXPORT_SYMBOL(vabits_actual);
-#endif
-
 u64 kimage_voffset __ro_after_init;
 EXPORT_SYMBOL(kimage_voffset);
 
arch/arm64/mm/proc.S +7 -9
index 98531775ff529dc8..02818fa6aded3218 100644
@@ -400,8 +400,6 @@ SYM_FUNC_END(idmap_kpti_install_ng_mappings)
  *
  *	Initialise the processor for turning the MMU on.
  *
- * Input:
- *	x0 - actual number of VA bits (ignored unless VA_BITS > 48)
  * Output:
  *	Return in x0 the value of the SCTLR_EL1 register.
  */
@@ -426,20 +424,20 @@ SYM_FUNC_START(__cpu_setup)
 	mair	.req	x17
 	tcr	.req	x16
 	mov_q	mair, MAIR_EL1_SET
-	mov_q	tcr, TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \
+	mov_q	tcr, TCR_TxSZ(VA_BITS_MIN) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \
 			TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \
 			TCR_TBI0 | TCR_A1 | TCR_KASAN_SW_FLAGS | TCR_MTE_FLAGS
 
 	tcr_clear_errata_bits tcr, x9, x5
 
-#ifdef CONFIG_ARM64_VA_BITS_52
-	sub		x9, xzr, x0
-	add		x9, x9, #64
-	tcr_set_t1sz	tcr, x9
-#else
+#if VA_BITS > VA_BITS_MIN
+alternative_if ARM64_HAS_LVA
+	eor		tcr, tcr, #TCR_T1SZ(VA_BITS) ^ TCR_T1SZ(VA_BITS_MIN)
+alternative_else_nop_endif
+#elif VA_BITS < 48
 	idmap_get_t0sz	x9
-#endif
 	tcr_set_t0sz	tcr, x9
+#endif
 
 	/*
 	 * Set the IPS bits in TCR_EL1.
arch/arm64/tools/cpucaps +1 -0
index f1c0347ec31a85c7..ec650a2cf4330179 100644
@@ -30,6 +30,7 @@ HAS_GENERIC_AUTH_IMP_DEF
 HAS_IRQ_PRIO_MASKING
 HAS_LDAPR
 HAS_LSE_ATOMICS
+HAS_LVA
 HAS_NO_FPSIMD
 HAS_NO_HW_PREFETCH
 HAS_PAN
Signature · 7 lines
2.35.1


_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
maintainer
#13 lore raw

Re: [PATCH 3/3] arm64: mm: Add feature override support for LVA and E0PD

In reply to Ard Biesheuvel
Quoted 6 lines · Ard Biesheuvel

Add support for overriding the VARange and E0PD fields of the MMFR2 CPU ID register. This permits the associated features to be overridden early enough for the boot code that creates the kernel mapping to take it into account.

Quoted 8 lines
---
 arch/arm64/include/asm/assembler.h    | 17 ++++++++++-------
 arch/arm64/include/asm/cpufeature.h   |  1 +
 arch/arm64/kernel/cpufeature.c        |  6 +++++-
 arch/arm64/kernel/image-vars.h        |  1 +
 arch/arm64/kernel/pi/idreg-override.c |  8 +++++++-
 arch/arm64/kernel/pi/map_kernel.c     |  4 ++++
 6 files changed, 28 insertions(+), 9 deletions(-)

What's the rationale for overriding E0PD? It seems unrelated to the rest of the series.

Will

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#14 lore raw

Re: [PATCH 3/3] arm64: mm: Add feature override support for LVA and E0PD

Quoted 17 lines · Will Deacon
Quoted 6 lines · Ard Biesheuvel

Add support for overriding the VARange and E0PD fields of the MMFR2 CPU ID register. This permits the associated features to be overridden early enough for the boot code that creates the kernel mapping to take it into account.

Quoted 8 lines
---
 arch/arm64/include/asm/assembler.h    | 17 ++++++++++-------
 arch/arm64/include/asm/cpufeature.h   |  1 +
 arch/arm64/kernel/cpufeature.c        |  6 +++++-
 arch/arm64/kernel/image-vars.h        |  1 +
 arch/arm64/kernel/pi/idreg-override.c |  8 +++++++-
 arch/arm64/kernel/pi/map_kernel.c     |  4 ++++
 6 files changed, 28 insertions(+), 9 deletions(-)
Quoted 3 lines

What's the rationale for overriding E0PD? It seems unrelated to the rest of the series.

It is not strictly related, but it is also taken into account by the early kernel mapping code and sits in the same ID register. I could imagine wanting to override it if it doesn't actually work (to avoid running KASLR without nG attributes) but I have no reason to assume someone actually needs that today.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

maintainer
#4 lore raw

Re: [PATCH 2/3] arm64: mm: Handle LVA support as a CPU feature

In reply to Ard Biesheuvel
Quoted 13 lines · Ard Biesheuvel

Currently, we detect CPU support for 52-bit virtual addressing (LVA) extremely early, before creating the kernel page tables or enabling the MMU. We cannot override the feature this early, and so large virtual addressing is always enabled on CPUs that implement support for it if the software support for it was enabled at build time. It also means we rely on non-trivial code in asm to deal with this feature.

Given that both the ID map and the TTBR1 mapping of the kernel image are guaranteed to be 48-bit addressable, it is not actually necessary to enable support this early, and instead, we can model it as a CPU feature. That way, we can rely on code patching to get the correct TCR.T1SZ values programmed on secondary boot and suspend from resume.

Quoted 5 lines

On the primary boot path, we simply enable the MMU with 48-bit virtual addressing initially, and update TCR.T1SZ from C code if LVA is supported, right before creating the kernel mapping. Given that TTBR1 still points to reserved_pg_dir at this point, updating TCR.T1SZ should be safe without the need for explicit TLB maintenance.

I'm not sure we can skip the TLBI here. There's some weird rule in the ARM ARM that if you change any of fields that may be cached in the TLB, maintenance is needed even if the MMU is off. From the latest version (I.a, I didn't dig into H.a),

R_VNRFW:
When a System register field is modified and that field is permitted to be cached in a TLB, software is required to invalidate all TLB entries that might be affected by the field, at any address translation stage in the translation regime even if the translation stage is disabled, using the appropriate VMID and ASID, after any required System register synchronization.

Signature · 6 lines
Catalin

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
maintainer Acked-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] arm64: mm: get rid of kimage_vaddr global variable

In reply to Ard Biesheuvel
Quoted 8 lines · Ard Biesheuvel

We store the address of _text in kimage_vaddr, but since commit 09e3c22a86f6889d ("arm64: Use a variable to store non-global mappings decision"), we no longer reference this variable from modules so we no longer need to export it.

In fact, we don't need it at all so let's just get rid of it.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#5 lore raw

Re: [PATCH 2/3] arm64: mm: Handle LVA support as a CPU feature

In reply to Catalin Marinas
Quoted 26 lines · Catalin Marinas
Quoted 13 lines · Ard Biesheuvel

Currently, we detect CPU support for 52-bit virtual addressing (LVA) extremely early, before creating the kernel page tables or enabling the MMU. We cannot override the feature this early, and so large virtual addressing is always enabled on CPUs that implement support for it if the software support for it was enabled at build time. It also means we rely on non-trivial code in asm to deal with this feature.

Given that both the ID map and the TTBR1 mapping of the kernel image are guaranteed to be 48-bit addressable, it is not actually necessary to enable support this early, and instead, we can model it as a CPU feature. That way, we can rely on code patching to get the correct TCR.T1SZ values programmed on secondary boot and suspend from resume.

Quoted 5 lines

On the primary boot path, we simply enable the MMU with 48-bit virtual addressing initially, and update TCR.T1SZ from C code if LVA is supported, right before creating the kernel mapping. Given that TTBR1 still points to reserved_pg_dir at this point, updating TCR.T1SZ should be safe without the need for explicit TLB maintenance.

I'm not sure we can skip the TLBI here. There's some weird rule in the ARM ARM that if you change any of fields that may be cached in the TLB, maintenance is needed even if the MMU is off. From the latest version (I.a, I didn't dig into H.a),

Quoted 8 lines

R_VNRFW:
When a System register field is modified and that field is permitted to be cached in a TLB, software is required to invalidate all TLB entries that might be affected by the field, at any address translation stage in the translation regime even if the translation stage is disabled, using the appropriate VMID and ASID, after any required System register synchronization.

Don't we already rely on this in cpu_set_default_tcr_t0sz() / cpu_set_idmap_tcr_t0sz() ?

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

maintainer
#6 lore raw

Re: [PATCH 2/3] arm64: mm: Handle LVA support as a CPU feature

Quoted 34 lines · Ard Biesheuvel
Quoted 25 lines · Catalin Marinas
Quoted 13 lines · Ard Biesheuvel

Currently, we detect CPU support for 52-bit virtual addressing (LVA) extremely early, before creating the kernel page tables or enabling the MMU. We cannot override the feature this early, and so large virtual addressing is always enabled on CPUs that implement support for it if the software support for it was enabled at build time. It also means we rely on non-trivial code in asm to deal with this feature.

Given that both the ID map and the TTBR1 mapping of the kernel image are guaranteed to be 48-bit addressable, it is not actually necessary to enable support this early, and instead, we can model it as a CPU feature. That way, we can rely on code patching to get the correct TCR.T1SZ values programmed on secondary boot and suspend from resume.

Quoted 5 lines

On the primary boot path, we simply enable the MMU with 48-bit virtual addressing initially, and update TCR.T1SZ from C code if LVA is supported, right before creating the kernel mapping. Given that TTBR1 still points to reserved_pg_dir at this point, updating TCR.T1SZ should be safe without the need for explicit TLB maintenance.

I'm not sure we can skip the TLBI here. There's some weird rule in the ARM ARM that if you change any of fields that may be cached in the TLB, maintenance is needed even if the MMU is off. From the latest version (I.a, I didn't dig into H.a),

Quoted 7 lines

R_VNRFW:
When a System register field is modified and that field is permitted to be cached in a TLB, software is required to invalidate all TLB entries that might be affected by the field, at any address translation stage in the translation regime even if the translation stage is disabled, using the appropriate VMID and ASID, after any required System register synchronization.

Quoted 2 lines

Don't we already rely on this in cpu_set_default_tcr_t0sz() / cpu_set_idmap_tcr_t0sz() ?

Yeah, we do this and depending on how you read the above rule, we may need to move the local_flush_tlb_all() line after T0SZ setting.

Signature · 6 lines
Catalin

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
#7 lore raw

Re: [PATCH 2/3] arm64: mm: Handle LVA support as a CPU feature

Quoted 36 lines · Catalin Marinas
Quoted 34 lines · Ard Biesheuvel
Quoted 32 lines · Catalin Marinas
Quoted 13 lines · Ard Biesheuvel

Currently, we detect CPU support for 52-bit virtual addressing (LVA) extremely early, before creating the kernel page tables or enabling the MMU. We cannot override the feature this early, and so large virtual addressing is always enabled on CPUs that implement support for it if the software support for it was enabled at build time. It also means we rely on non-trivial code in asm to deal with this feature.

Given that both the ID map and the TTBR1 mapping of the kernel image are guaranteed to be 48-bit addressable, it is not actually necessary to enable support this early, and instead, we can model it as a CPU feature. That way, we can rely on code patching to get the correct TCR.T1SZ values programmed on secondary boot and suspend from resume.

Quoted 5 lines

On the primary boot path, we simply enable the MMU with 48-bit virtual addressing initially, and update TCR.T1SZ from C code if LVA is supported, right before creating the kernel mapping. Given that TTBR1 still points to reserved_pg_dir at this point, updating TCR.T1SZ should be safe without the need for explicit TLB maintenance.

I'm not sure we can skip the TLBI here. There's some weird rule in the ARM ARM that if you change any of fields that may be cached in the TLB, maintenance is needed even if the MMU is off. From the latest version (I.a, I didn't dig into H.a),

R_VNRFW:
When a System register field is modified and that field is permitted to be cached in a TLB, software is required to invalidate all TLB entries that might be affected by the field, at any address translation stage in the translation regime even if the translation stage is disabled, using the appropriate VMID and ASID, after any required System register synchronization.

Quoted 6 lines
Quoted 2 lines

Don't we already rely on this in cpu_set_default_tcr_t0sz() / cpu_set_idmap_tcr_t0sz() ?

Yeah, we do this and depending on how you read the above rule, we may need to move the local_flush_tlb_all() line after T0SZ setting.

OK, so wouldn't this mean that we cannot change TxSZ at all without going through a MMU off/on cycle?

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

maintainer
#8 lore raw

Re: [PATCH 2/3] arm64: mm: Handle LVA support as a CPU feature

Quoted 36 lines · Ard Biesheuvel
Quoted 35 lines · Catalin Marinas
Quoted 34 lines · Ard Biesheuvel
Quoted 32 lines · Catalin Marinas
Quoted 13 lines · Ard Biesheuvel

Currently, we detect CPU support for 52-bit virtual addressing (LVA) extremely early, before creating the kernel page tables or enabling the MMU. We cannot override the feature this early, and so large virtual addressing is always enabled on CPUs that implement support for it if the software support for it was enabled at build time. It also means we rely on non-trivial code in asm to deal with this feature.

Given that both the ID map and the TTBR1 mapping of the kernel image are guaranteed to be 48-bit addressable, it is not actually necessary to enable support this early, and instead, we can model it as a CPU feature. That way, we can rely on code patching to get the correct TCR.T1SZ values programmed on secondary boot and suspend from resume.

Quoted 5 lines

On the primary boot path, we simply enable the MMU with 48-bit virtual addressing initially, and update TCR.T1SZ from C code if LVA is supported, right before creating the kernel mapping. Given that TTBR1 still points to reserved_pg_dir at this point, updating TCR.T1SZ should be safe without the need for explicit TLB maintenance.

I'm not sure we can skip the TLBI here. There's some weird rule in the ARM ARM that if you change any of fields that may be cached in the TLB, maintenance is needed even if the MMU is off. From the latest version (I.a, I didn't dig into H.a),

R_VNRFW:
When a System register field is modified and that field is permitted to be cached in a TLB, software is required to invalidate all TLB entries that might be affected by the field, at any address translation stage in the translation regime even if the translation stage is disabled, using the appropriate VMID and ASID, after any required System register synchronization.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Don't we already rely on this in cpu_set_default_tcr_t0sz() / cpu_set_idmap_tcr_t0sz() ?

Yeah, we do this and depending on how you read the above rule, we may need to move the local_flush_tlb_all() line after T0SZ setting.

OK, so wouldn't this mean that we cannot change TxSZ at all without going through a MMU off/on cycle?

I don't think so. The way I see it is that the change is not guaranteed to take effect until we invalidate the TLBs. We don't risk fetching random stuff in the TLB since we have the reserved TTBR0 at that point. If the subsequent cpu_switch_mm() changed the ASID, in some interpretation of the ARM ARM we could have skipped the TLBI but that's not the case anyway.

Looking for Mark R's opinion as I recall he talked to the architects in the past about this (though I think it was in the context of SCTLR_EL1).

Signature · 6 lines
Catalin

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
reviewer
#9 lore raw

Re: [PATCH 2/3] arm64: mm: Handle LVA support as a CPU feature

Quoted 53 lines · Catalin Marinas
Quoted 36 lines · Ard Biesheuvel
Quoted 35 lines · Catalin Marinas
Quoted 34 lines · Ard Biesheuvel
Quoted 32 lines · Catalin Marinas
Quoted 13 lines · Ard Biesheuvel

Currently, we detect CPU support for 52-bit virtual addressing (LVA) extremely early, before creating the kernel page tables or enabling the MMU. We cannot override the feature this early, and so large virtual addressing is always enabled on CPUs that implement support for it if the software support for it was enabled at build time. It also means we rely on non-trivial code in asm to deal with this feature.

Given that both the ID map and the TTBR1 mapping of the kernel image are guaranteed to be 48-bit addressable, it is not actually necessary to enable support this early, and instead, we can model it as a CPU feature. That way, we can rely on code patching to get the correct TCR.T1SZ values programmed on secondary boot and suspend from resume.

Quoted 5 lines

On the primary boot path, we simply enable the MMU with 48-bit virtual addressing initially, and update TCR.T1SZ from C code if LVA is supported, right before creating the kernel mapping. Given that TTBR1 still points to reserved_pg_dir at this point, updating TCR.T1SZ should be safe without the need for explicit TLB maintenance.

I'm not sure we can skip the TLBI here. There's some weird rule in the ARM ARM that if you change any of fields that may be cached in the TLB, maintenance is needed even if the MMU is off. From the latest version (I.a, I didn't dig into H.a),

R_VNRFW:
When a System register field is modified and that field is permitted to be cached in a TLB, software is required to invalidate all TLB entries that might be affected by the field, at any address translation stage in the translation regime even if the translation stage is disabled, using the appropriate VMID and ASID, after any required System register synchronization.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Don't we already rely on this in cpu_set_default_tcr_t0sz() / cpu_set_idmap_tcr_t0sz() ?

Yeah, we do this and depending on how you read the above rule, we may need to move the local_flush_tlb_all() line after T0SZ setting.

OK, so wouldn't this mean that we cannot change TxSZ at all without going through a MMU off/on cycle?

I don't think so. The way I see it is that the change is not guaranteed to take effect until we invalidate the TLBs. We don't risk fetching random stuff in the TLB since we have the reserved TTBR0 at that point. If the subsequent cpu_switch_mm() changed the ASID, in some interpretation of the ARM ARM we could have skipped the TLBI but that's not the case anyway.

Quoted 2 lines

Looking for Mark R's opinion as I recall he talked to the architects in the past about this (though I think it was in the context of SCTLR_EL1).

The architecture is unfortunately vague here.

From prior discussions with architects, the general rule was "if it's permitted to be cached in a TLB, an update must be followed by a TLBI for that update to take effect, regardless of SCTLR_ELx.M". The architecture isn't very specific about what scope fo maintenance is required (e.g. if certain fields are tagged by ASID/VMID), but I beleive it's sufficient to use a (VM)ALL for the current translation regime (which might be stronger than necessary).

So for this case, my understanding is:

1) When changing TxSZ, we need a subsequent invalidate before the change is guaranteed to take effect. So I believe that what we do today isn't quite right.

2) During the window between writing to TxSZ and completing the invalidation, I'm not sure how the MMU is permitted to behave w.r.t. intermediate walk entries. I could imagine that those become (CONSTRAINED) UNPREDICTABLE , and that we might need to ensure those are invalidated (or inactive and prior walks completed) before we write to TCR_EL1.

I can go chase this up with the architects; in the mean time my thinking would be that we should retain the existing maintenance.

There's almost certainly more stuff that we'd need to fix in this area.

Thanks,
Mark.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

#10 lore raw

Re: [PATCH 2/3] arm64: mm: Handle LVA support as a CPU feature

Quoted 73 lines · Mark Rutland
Quoted 53 lines · Catalin Marinas
Quoted 36 lines · Ard Biesheuvel
Quoted 35 lines · Catalin Marinas
Quoted 34 lines · Ard Biesheuvel
Quoted 32 lines · Catalin Marinas
Quoted 13 lines · Ard Biesheuvel

Currently, we detect CPU support for 52-bit virtual addressing (LVA) extremely early, before creating the kernel page tables or enabling the MMU. We cannot override the feature this early, and so large virtual addressing is always enabled on CPUs that implement support for it if the software support for it was enabled at build time. It also means we rely on non-trivial code in asm to deal with this feature.

Given that both the ID map and the TTBR1 mapping of the kernel image are guaranteed to be 48-bit addressable, it is not actually necessary to enable support this early, and instead, we can model it as a CPU feature. That way, we can rely on code patching to get the correct TCR.T1SZ values programmed on secondary boot and suspend from resume.

Quoted 5 lines

On the primary boot path, we simply enable the MMU with 48-bit virtual addressing initially, and update TCR.T1SZ from C code if LVA is supported, right before creating the kernel mapping. Given that TTBR1 still points to reserved_pg_dir at this point, updating TCR.T1SZ should be safe without the need for explicit TLB maintenance.

I'm not sure we can skip the TLBI here. There's some weird rule in the ARM ARM that if you change any of fields that may be cached in the TLB, maintenance is needed even if the MMU is off. From the latest version (I.a, I didn't dig into H.a),

R_VNRFW:
When a System register field is modified and that field is permitted to be cached in a TLB, software is required to invalidate all TLB entries that might be affected by the field, at any address translation stage in the translation regime even if the translation stage is disabled, using the appropriate VMID and ASID, after any required System register synchronization.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Don't we already rely on this in cpu_set_default_tcr_t0sz() / cpu_set_idmap_tcr_t0sz() ?

Yeah, we do this and depending on how you read the above rule, we may need to move the local_flush_tlb_all() line after T0SZ setting.

OK, so wouldn't this mean that we cannot change TxSZ at all without going through a MMU off/on cycle?

I don't think so. The way I see it is that the change is not guaranteed to take effect until we invalidate the TLBs. We don't risk fetching random stuff in the TLB since we have the reserved TTBR0 at that point. If the subsequent cpu_switch_mm() changed the ASID, in some interpretation of the ARM ARM we could have skipped the TLBI but that's not the case anyway.

Quoted 2 lines

Looking for Mark R's opinion as I recall he talked to the architects in the past about this (though I think it was in the context of SCTLR_EL1).

The architecture is unfortunately vague here.

From prior discussions with architects, the general rule was "if it's permitted to be cached in a TLB, an update must be followed by a TLBI for that update to take effect, regardless of SCTLR_ELx.M". The architecture isn't very specific about what scope fo maintenance is required (e.g. if certain fields are tagged by ASID/VMID), but I beleive it's sufficient to use a (VM)ALL for the current translation regime (which might be stronger than necessary).

So for this case, my understanding is:

1) When changing TxSZ, we need a subsequent invalidate before the change is guaranteed to take effect. So I believe that what we do today isn't quite right.

Quoted 6 lines

2) During the window between writing to TxSZ and completing the invalidation, I'm not sure how the MMU is permitted to behave w.r.t. intermediate walk entries. I could imagine that those become (CONSTRAINED) UNPREDICTABLE , and that we might need to ensure those are invalidated (or inactive and prior walks completed) before we write to TCR_EL1.

OK, so that would at least mean that we can modify T1SZ while TTBR1 is pointing to reserved_pg_dir without running the risk of TLB conflicts, right?
(and similarly for TTBR0). Given that we never modify TxSZ on a hot
path, sprinkling some bonus TLBIs (and pivoting via reserved_pg_dir if needed) is hardly an issue here.

Quoted 3 lines

I can go chase this up with the architects; in the mean time my thinking would be that we should retain the existing maintenance.

Yes, that would be helpful, thanks.

Quoted 4 lines

There's almost certainly more stuff that we'd need to fix in this area.

Thanks,
Mark.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

reviewer
#11 lore raw

Re: [PATCH 2/3] arm64: mm: Handle LVA support as a CPU feature

Quoted 81 lines · Ard Biesheuvel
Quoted 73 lines · Mark Rutland
Quoted 53 lines · Catalin Marinas
Quoted 36 lines · Ard Biesheuvel
Quoted 35 lines · Catalin Marinas
Quoted 34 lines · Ard Biesheuvel
Quoted 32 lines · Catalin Marinas
Quoted 13 lines · Ard Biesheuvel

Currently, we detect CPU support for 52-bit virtual addressing (LVA) extremely early, before creating the kernel page tables or enabling the MMU. We cannot override the feature this early, and so large virtual addressing is always enabled on CPUs that implement support for it if the software support for it was enabled at build time. It also means we rely on non-trivial code in asm to deal with this feature.

Given that both the ID map and the TTBR1 mapping of the kernel image are guaranteed to be 48-bit addressable, it is not actually necessary to enable support this early, and instead, we can model it as a CPU feature. That way, we can rely on code patching to get the correct TCR.T1SZ values programmed on secondary boot and suspend from resume.

Quoted 5 lines

On the primary boot path, we simply enable the MMU with 48-bit virtual addressing initially, and update TCR.T1SZ from C code if LVA is supported, right before creating the kernel mapping. Given that TTBR1 still points to reserved_pg_dir at this point, updating TCR.T1SZ should be safe without the need for explicit TLB maintenance.

I'm not sure we can skip the TLBI here. There's some weird rule in the ARM ARM that if you change any of fields that may be cached in the TLB, maintenance is needed even if the MMU is off. From the latest version (I.a, I didn't dig into H.a),

R_VNRFW:
When a System register field is modified and that field is permitted to be cached in a TLB, software is required to invalidate all TLB entries that might be affected by the field, at any address translation stage in the translation regime even if the translation stage is disabled, using the appropriate VMID and ASID, after any required System register synchronization.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Don't we already rely on this in cpu_set_default_tcr_t0sz() / cpu_set_idmap_tcr_t0sz() ?

Yeah, we do this and depending on how you read the above rule, we may need to move the local_flush_tlb_all() line after T0SZ setting.

OK, so wouldn't this mean that we cannot change TxSZ at all without going through a MMU off/on cycle?

I don't think so. The way I see it is that the change is not guaranteed to take effect until we invalidate the TLBs. We don't risk fetching random stuff in the TLB since we have the reserved TTBR0 at that point. If the subsequent cpu_switch_mm() changed the ASID, in some interpretation of the ARM ARM we could have skipped the TLBI but that's not the case anyway.

Quoted 2 lines

Looking for Mark R's opinion as I recall he talked to the architects in the past about this (though I think it was in the context of SCTLR_EL1).

The architecture is unfortunately vague here.

From prior discussions with architects, the general rule was "if it's permitted to be cached in a TLB, an update must be followed by a TLBI for that update to take effect, regardless of SCTLR_ELx.M". The architecture isn't very specific about what scope fo maintenance is required (e.g. if certain fields are tagged by ASID/VMID), but I beleive it's sufficient to use a (VM)ALL for the current translation regime (which might be stronger than necessary).

So for this case, my understanding is:

1) When changing TxSZ, we need a subsequent invalidate before the change is guaranteed to take effect. So I believe that what we do today isn't quite right.

Quoted 6 lines

2) During the window between writing to TxSZ and completing the invalidation, I'm not sure how the MMU is permitted to behave w.r.t. intermediate walk entries. I could imagine that those become (CONSTRAINED) UNPREDICTABLE , and that we might need to ensure those are invalidated (or inactive and prior walks completed) before we write to TCR_EL1.

Quoted 3 lines

OK, so that would at least mean that we can modify T1SZ while TTBR1 is pointing to reserved_pg_dir without running the risk of TLB conflicts, right? (and similarly for TTBR0).

That is my understanding, yes.

Quoted 2 lines

Given that we never modify TxSZ on a hot path, sprinkling some bonus TLBIs (and pivoting via reserved_pg_dir if needed) is hardly an issue here.

Agreed.

Quoted 4 lines
Quoted 2 lines

I can go chase this up with the architects; in the mean time my thinking would be that we should retain the existing maintenance.

Yes, that would be helpful, thanks.

Cool; I\ll try to do that shortly, and will keep you updated.

Thanks,
Mark.

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel