arm64/kernel: Always use level 2 or higher for early mappings

From: Ard Biesheuvel <[email protected]>

The page table population code in map_range() uses a recursive algorithm to create the early mappings of the kernel, the DTB and the ID mapped text and data pages, and this fails to take into account that the way these page tables may be constructed is not precisely the same at each level. In particular, block mappings are not permitted at each level, and the code as it exists today might inadvertently create such a forbidden block mapping if it were used to map a region of the appropriate size and alignment.

This never happens in practice, given the limited size of the assets being mapped by the early boot code. Nonetheless, it would be better if this code would behave correctly in all circumstances.

So only permit block mappings at level 2, and page mappings at level 3, for any page size, and use table mappings exclusively at all other levels. This change should have no impact in practice, but it makes the code more robust.

Cc Anshuman Khandual <[email protected]>

Reported-by Ryan Roberts <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

---
 arch/arm64/kernel/pi/map_range.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

arch/arm64/kernel/pi/map_range.c +9 -8

index 2b69e3beeef8..025bb9f0aa0b 100644
@@ -40,17 +40,10 @@ void __init map_range(u64 *pte, u64 start, u64 end, u64 pa, pgprot_t prot,
 	/* Advance tbl to the entry that covers start */
 	tbl += (start >> (lshift + PAGE_SHIFT)) % PTRS_PER_PTE;
 
-	/*
-	 * Set the right block/page bits for this level unless we are
-	 * clearing the mapping
-	 */
-	if (protval)
-		protval |= (level < 3) ? PMD_TYPE_SECT : PTE_TYPE_PAGE;
-
 	while (start < end) {
 		u64 next = min((start | lmask) + 1, PAGE_ALIGN(end));
 
-		if (level < 3 && (start | next | pa) & lmask) {
+		if (level < 2 || (level == 2 && (start | next | pa) & lmask)) {
 			/*
 			 * This chunk needs a finer grained mapping. Create a
 			 * table mapping if necessary and recurse.

@@ -64,6 +57,14 @@ void __init map_range(u64 *pte, u64 start, u64 end, u64 pa, pgprot_t prot,
 				  (pte_t *)(__pte_to_phys(*tbl) + va_offset),
 				  may_use_cont, va_offset);
 		} else {
+			/*
+			 * Set the right block/page bits for this level unless
+			 * we are clearing the mapping
+			 */
+			if (protval)
+				protval |= (level == 2) ? PMD_TYPE_SECT
+							: PTE_TYPE_PAGE;
+
 			/*
 			 * Start a contiguous range if start and pa are
 			 * suitably aligned

From: Ard Biesheuvel <[email protected]>

The page table population code in map_range() uses a recursive algorithm to create the early mappings of the kernel, the DTB and the ID mapped text and data pages, and this fails to take into account that the way these page tables may be constructed is not precisely the same at each level. In particular, block mappings are not permitted at each level, and the code as it exists today might inadvertently create such a forbidden block mapping if it were used to map a region of the appropriate size and alignment.

This never happens in practice, given the limited size of the assets being mapped by the early boot code. Nonetheless, it would be better if this code would behave correctly in all circumstances.

So only permit block mappings at level 2, and page mappings at level 3, for any page size, and use table mappings exclusively at all other levels. This change should have no impact in practice, but it makes the code more robust.

Cc Anshuman Khandual <[email protected]>

Reported-by Ryan Roberts <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

---
 arch/arm64/kernel/pi/map_range.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

arch/arm64/kernel/pi/map_range.c +2 -8

index 2b69e3beeef8..025bb9f0aa0b 100644
@@ -40,17 +40,10 @@ void __init map_range(u64 *pte, u64 start, u64 end, u64 pa, pgprot_t prot,
 	/* Advance tbl to the entry that covers start */
 	tbl += (start >> (lshift + PAGE_SHIFT)) % PTRS_PER_PTE;
 
-	/*
-	 * Set the right block/page bits for this level unless we are
-	 * clearing the mapping
-	 */
-	if (protval)
-		protval |= (level < 3) ? PMD_TYPE_SECT : PTE_TYPE_PAGE;
-
 	while (start < end) {
 		u64 next = min((start | lmask) + 1, PAGE_ALIGN(end));
 
-		if (level < 3 && (start | next | pa) & lmask) {
+		if (level < 2 || (level == 2 && (start | next | pa) & lmask)) {
 			/*
 			 * This chunk needs a finer grained mapping. Create a
 			 * table mapping if necessary and recurse.

@@ -64,6 +57,14 @@ void __init map_range(u64 *pte, u64 start, u64 end, u64 pa, pgprot_t prot,
 				  (pte_t *)(__pte_to_phys(*tbl) + va_offset),
 				  may_use_cont, va_offset);
 		} else {
+			/*

Quoted 55 lines · Ard Biesheuvel

From: Ard Biesheuvel <[email protected]>

The page table population code in map_range() uses a recursive algorithm to create the early mappings of the kernel, the DTB and the ID mapped text and data pages, and this fails to take into account that the way these page tables may be constructed is not precisely the same at each level. In particular, block mappings are not permitted at each level, and the code as it exists today might inadvertently create such a forbidden block mapping if it were used to map a region of the appropriate size and alignment.

This never happens in practice, given the limited size of the assets being mapped by the early boot code. Nonetheless, it would be better if this code would behave correctly in all circumstances.

So only permit block mappings at level 2, and page mappings at level 3, for any page size, and use table mappings exclusively at all other levels. This change should have no impact in practice, but it makes the code more robust.

Cc Anshuman Khandual <[email protected]>

Reported-by Ryan Roberts <[email protected]>

Signed-off-by Ard Biesheuvel <[email protected]>

---
 arch/arm64/kernel/pi/map_range.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

arch/arm64/kernel/pi/map_range.c +2 -8

index 2b69e3beeef8..025bb9f0aa0b 100644
@@ -40,17 +40,10 @@ void __init map_range(u64 *pte, u64 start, u64 end, u64 pa, pgprot_t prot,
      /* Advance tbl to the entry that covers start */
      tbl += (start >> (lshift + PAGE_SHIFT)) % PTRS_PER_PTE;
 
-     /*
-      * Set the right block/page bits for this level unless we are
-      * clearing the mapping
-      */
-     if (protval)
-             protval |= (level < 3) ? PMD_TYPE_SECT : PTE_TYPE_PAGE;
-
      while (start < end) {
              u64 next = min((start | lmask) + 1, PAGE_ALIGN(end));
 
-             if (level < 3 && (start | next | pa) & lmask) {
+             if (level < 2 || (level == 2 && (start | next | pa) & lmask)) {
                      /*
                       * This chunk needs a finer grained mapping. Create a
                       * table mapping if necessary and recurse.

@@ -64,6 +57,14 @@ void __init map_range(u64 *pte, u64 start, u64 end, u64 pa, pgprot_t prot,
                                (pte_t *)(__pte_to_phys(*tbl) + va_offset),
                                may_use_cont, va_offset);
              } else {
+                     /*

arch/arm64/kernel/pi/map_range.c void __init map_range(u64 *pte, u64 start, u64 end, u64 pa, pgprot_t prot, · line 70

Quoted 10 lines

arch/arm64/kernel/pi/map_range.c +7 -0

+                      * Set the right block/page bits for this level unless
+                      * we are clearing the mapping
+                      */
+                     if (protval)
+                             protval |= (level == 2) ? PMD_TYPE_SECT
+                                                     : PTE_TYPE_PAGE;
+
                      /*
                       * Start a contiguous range if start and pa are
                       * suitably aligned

LGTM and as Ryan mentioned, D128 ready as well !

Reviewed-by Anshuman Khandual <[email protected]>