Skip to contentSkip to conversation

arm: Fix cache inconsistency when using fixmap

RFC PATCH New Jon Medhurst (Tixy) 3 participants 9 messages Feb 22, 2017 – Feb 27, 2017 Arm (32-bit)

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 arm: Fix cache inconsistency when using fixmap

Cacheable memory mappings need to be marked as shareable otherwise the CPU accessing fixmap memory may end up with local cachelines for that memory, resulting in different CPUs in the system seeing different values for the same memory location.

This issue was discovered when investigating failures in the ARM kprobe tests. kprobes uses patch_text() to modify the kernel image, which when CONFIG_DEBUG_RODATA is enabled makes use of fixmap to map the page to be modified. As the shareable attribute wasn't being set in the PTE entry for that page, a write to it wasn't being broadcast to other caches in the system, which in the the case being investigated was CPU's in the other cluster of a big.LITTLE system.

Fixes a5f4c561b3b1 ("ARM: 8415/1: early fixmap support for earlycon")
Cc stable at vger.kernel.org # v4.3+

---

The fixmap changes in the original commit a5f4c561b3b1 look a bit iffy to me, shouldn't it have have made use of PAGE_KERNEL or pgprot_kernel or something like that to get PTE attributes suitable for the system being run, rather than rolling it's own set of values? That's why I'm marking this as an RFC, perhaps the correct fix is to rework the code to use the defines from arch/arm/include/asm/pgtable.h

 arch/arm/include/asm/fixmap.h       | 2 +-
 arch/arm/probes/kprobes/test-core.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/arm/include/asm/fixmap.h +1 -1
index 5c17d2dec777..26d4a4677cd7 100644
@@ -41,7 +41,7 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON	(L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL	(FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL	(FIXMAP_PAGE_COMMON | L_PTE_SHARED | L_PTE_MT_WRITEBACK)
 #define FIXMAP_PAGE_RO		(FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
arch/arm/probes/kprobes/test-core.h +1 -1
index 94285203e9f7..cde2b4e9358a 100644
@@ -8,7 +8,7 @@
  * published by the Free Software Foundation.
  */
 
-#define VERBOSE 0 /* Set to '1' for more logging of test cases */
+#define VERBOSE 1 /* Set to '1' for more logging of test cases */
 
 #ifdef CONFIG_THUMB2_KERNEL
 #define NORMAL_ISA "16"
Signature · 1 line
2.11.0
#1 lore raw

RFC PATCH 1/1 arm: Fix cache inconsistency when using fixmap

[also added Ard]

Quoted 16 lines · Jon Medhurst (Tixy)

Cacheable memory mappings need to be marked as shareable otherwise the CPU accessing fixmap memory may end up with local cachelines for that memory, resulting in different CPUs in the system seeing different values for the same memory location.

This issue was discovered when investigating failures in the ARM kprobe tests. kprobes uses patch_text() to modify the kernel image, which when CONFIG_DEBUG_RODATA is enabled makes use of fixmap to map the page to be modified. As the shareable attribute wasn't being set in the PTE entry for that page, a write to it wasn't being broadcast to other caches in the system, which in the the case being investigated was CPU's in the other cluster of a big.LITTLE system.

Fixes a5f4c561b3b1 ("ARM: 8415/1: early fixmap support for earlycon")
Cc stable at vger.kernel.org # v4.3+
Quoted 7 lines

---

The fixmap changes in the original commit a5f4c561b3b1 look a bit iffy to me, shouldn't it have have made use of PAGE_KERNEL or pgprot_kernel or something like that to get PTE attributes suitable for the system being run, rather than rolling it's own set of values?

The PAGE_KERNEL points to pgprot_kernel, which in turn is dynamically setup in build_mem_type_table. This initialization is taking place late in the boot process, too late for early fixmap...

Early fixmap gets shut down before going SMP, so non shared mappings are not an issue there. But FIXMAP_PAGE_NORMAL is also used for regular fixmap, which is essentially the problem. Also, regular fixmap does not make sure of the cache policy due to that, but just uses the hardcoded writeback policy...

With that in mind, populating kern_pgprot with a reasonable default which works for early fixmap might be a reasonable approach. Later kern_pgprot gets setup with the appropriate shared attribute and cache policy...

(quoted diff) +0 -0

b/arch/arm/include/asm/fixmap.h
index 5c17d2dec777..c663e562716c 100644

arch/arm/include/asm/fixmap.h +0 -0
@@ -39,12 +39,11 @@ static const enum fixed_addresses

__end_of_fixed_addresses =
__end_of_fixmap_region > __end_of_early_ioremap_region ? __end_of_fixmap_region : __end_of_early_ioremap_region;

-#define FIXMAP_PAGE_COMMON (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN
| L_PTE_DIRTY)
-
-#define FIXMAP_PAGE_NORMAL (FIXMAP_PAGE_COMMON |
L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL (pgprot_kernel | L_PTE_XN)
#define FIXMAP_PAGE_RO (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)

/* Used by set_fixmap_(io|nocache), both meant for mapping a device */
+#define FIXMAP_PAGE_COMMON (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN
| L_PTE_DIRTY)
#define FIXMAP_PAGE_IO (FIXMAP_PAGE_COMMON |
L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

arch/arm/mm/mmu.c +2 -1
index 4001dd15818d..9cb7c559263f 100644
@@ -65,7 +65,8 @@ pmdval_t user_pmd_table = _PAGE_USER_TABLE;
 static unsigned int cachepolicy __initdata = CPOLICY_WRITEBACK;
 static unsigned int ecc_mask __initdata = 0;
 pgprot_t pgprot_user;
-pgprot_t pgprot_kernel;
+pgprot_t pgprot_kernel = __pgprot(L_PTE_PRESENT | L_PTE_YOUNG |
+                                 L_PTE_DIRTY | L_PTE_MT_WRITEBACK);
 pgprot_t pgprot_hyp_device;
 pgprot_t pgprot_s2;
 pgprot_t pgprot_s2_device;
Quoted 28 lines

That's why I'm marking this as an RFC, perhaps the correct fix is to rework the code to use the defines from arch/arm/include/asm/pgtable.h

 arch/arm/include/asm/fixmap.h       | 2 +-
 arch/arm/probes/kprobes/test-core.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/arm/include/asm/fixmap.h +1 -1
index 5c17d2dec777..26d4a4677cd7 100644
@@ -41,7 +41,7 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON	(L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN |
 L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL	(FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL	(FIXMAP_PAGE_COMMON | L_PTE_SHARED |
 L_PTE_MT_WRITEBACK)
 #define FIXMAP_PAGE_RO		(FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)

/* Used by set_fixmap_(io|nocache), both meant for mapping a device */

arch/arm/probes/kprobes/test-core.h +0 -0
index 94285203e9f7..cde2b4e9358a 100644

arch/arm/probes/kprobes/test-core.h line 11

Quoted 5 lines
arch/arm/probes/kprobes/test-core.h +1 -1
@@ -8,7 +8,7 @@
  * published by the Free Software Foundation.
  */
 
-#define VERBOSE 0 /* Set to '1' for more logging of test cases */
+#define VERBOSE 1 /* Set to '1' for more logging of test cases */

That seems to be unrelated.

--
Stefan

Quoted 3 lines
arch/arm/probes/kprobes/test-core.h +0 -0
 #ifdef CONFIG_THUMB2_KERNEL
 #define NORMAL_ISA "16"
#2 lore raw

RFC PATCH 1/1 arm: Fix cache inconsistency when using fixmap

Quoted 2 lines · Stefan Agner

[also added Ard]

Thanks

Quoted 71 lines · Jon Medhurst (Tixy)
Quoted 16 lines

Cacheable memory mappings need to be marked as shareable otherwise the CPU accessing fixmap memory may end up with local cachelines for that memory, resulting in different CPUs in the system seeing different values for the same memory location.

This issue was discovered when investigating failures in the ARM kprobe tests. kprobes uses patch_text() to modify the kernel image, which when CONFIG_DEBUG_RODATA is enabled makes use of fixmap to map the page to be modified. As the shareable attribute wasn't being set in the PTE entry for that page, a write to it wasn't being broadcast to other caches in the system, which in the the case being investigated was CPU's in the other cluster of a big.LITTLE system.

Fixes a5f4c561b3b1 ("ARM: 8415/1: early fixmap support for earlycon")
Cc stable at vger.kernel.org # v4.3+
Quoted 7 lines

---

The fixmap changes in the original commit a5f4c561b3b1 look a bit iffy to me, shouldn't it have have made use of PAGE_KERNEL or pgprot_kernel or something like that to get PTE attributes suitable for the system being run, rather than rolling it's own set of values?

The PAGE_KERNEL points to pgprot_kernel, which in turn is dynamically setup in build_mem_type_table. This initialization is taking place late in the boot process, too late for early fixmap...

Early fixmap gets shut down before going SMP, so non shared mappings are not an issue there. But FIXMAP_PAGE_NORMAL is also used for regular fixmap, which is essentially the problem. Also, regular fixmap does not make sure of the cache policy due to that, but just uses the hardcoded writeback policy...

With that in mind, populating kern_pgprot with a reasonable default which works for early fixmap might be a reasonable approach. Later kern_pgprot gets setup with the appropriate shared attribute and cache policy...

arch/arm/include/asm/fixmap.h +0 -0
index 5c17d2dec777..c663e562716c 100644
@@ -39,12 +39,11 @@ static const enum fixed_addresses

__end_of_fixed_addresses =
__end_of_fixmap_region > __end_of_early_ioremap_region ? __end_of_fixmap_region : __end_of_early_ioremap_region;

-#define FIXMAP_PAGE_COMMON (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN
| L_PTE_DIRTY)

arch/arm/include/asm/fixmap.h +0 -2
-
-#define FIXMAP_PAGE_NORMAL     (FIXMAP_PAGE_COMMON |

L_PTE_MT_WRITEBACK)

arch/arm/include/asm/fixmap.h +2 -0
+#define FIXMAP_PAGE_NORMAL     (pgprot_kernel | L_PTE_XN)
 #define FIXMAP_PAGE_RO         (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
+#define FIXMAP_PAGE_COMMON     (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN

| L_PTE_DIRTY)
#define FIXMAP_PAGE_IO (FIXMAP_PAGE_COMMON |
L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

arch/arm/mm/mmu.c +0 -0
index 4001dd15818d..9cb7c559263f 100644

arch/arm/mm/mmu.c pmdval_t user_pmd_table = _PAGE_USER_TABLE; · line 72

Quoted 9 lines
arch/arm/mm/mmu.c +2 -1
@@ -65,7 +65,8 @@ pmdval_t user_pmd_table = _PAGE_USER_TABLE;
 static unsigned int cachepolicy __initdata = CPOLICY_WRITEBACK;
 static unsigned int ecc_mask __initdata = 0;
 pgprot_t pgprot_user;
-pgprot_t pgprot_kernel;
+pgprot_t pgprot_kernel = __pgprot(L_PTE_PRESENT | L_PTE_YOUNG |
+                                 L_PTE_DIRTY | L_PTE_MT_WRITEBACK);
 pgprot_t pgprot_hyp_device;
 pgprot_t pgprot_s2;
 pgprot_t pgprot_s2_device;

This looks correct to me, and avoids using shareable attributes inadvertently.

However, we could make this a bit more private to the fixmap code by using something like

arch/arm/include/asm/fixmap.h +0 -0
index 5c17d2dec777..6c375aea8f22 100644
@@ -41,11 +41,14 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON     (L_PTE_YOUNG | L_PTE_PRESENT |

L_PTE_XN | L_PTE_DIRTY)

-#define FIXMAP_PAGE_NORMAL     (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO         (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define __FIXMAP_MT_NORMAL     ((pgprot_val(pgprot_kernel) ?:

FIXMAP_PAGE_COMMON) | \
+ L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL __pgprot(__FIXMAP_MT_NORMAL)
+#define FIXMAP_PAGE_RO __pgprot(__FIXMAP_MT_NORMAL | L_PTE_RDONLY)

/* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO (FIXMAP_PAGE_COMMON |
L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
+#define FIXMAP_PAGE_IO __pgprot(FIXMAP_PAGE_COMMON |
L_PTE_MT_DEV_SHARED | \
+ L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

#define __early_set_fixmap __set_fixmap

(and drop the change to mm/mmu.c), which boils down to the same for fixmap but does not affect other users of pgprot_kernel. Also, it appears these definitions are broken under STRICT_MM_TYPECHECKS so this is a good opportunity to get that fixed as well.

Thank,
Ard.

Quoted 44 lines
Quoted 30 lines

That's why I'm marking this as an RFC, perhaps the correct fix is to rework the code to use the defines from arch/arm/include/asm/pgtable.h

 arch/arm/include/asm/fixmap.h       | 2 +-
 arch/arm/probes/kprobes/test-core.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/arm/include/asm/fixmap.h +1 -1
index 5c17d2dec777..26d4a4677cd7 100644
@@ -41,7 +41,7 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON   (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN |
 L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL   (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL   (FIXMAP_PAGE_COMMON | L_PTE_SHARED |
 L_PTE_MT_WRITEBACK)
 #define FIXMAP_PAGE_RO               (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)

/* Used by set_fixmap_(io|nocache), both meant for mapping a device */

arch/arm/probes/kprobes/test-core.h +0 -0
index 94285203e9f7..cde2b4e9358a 100644

arch/arm/probes/kprobes/test-core.h line 11

Quoted 5 lines
arch/arm/probes/kprobes/test-core.h +1 -1
@@ -8,7 +8,7 @@
  * published by the Free Software Foundation.
  */
 
-#define VERBOSE 0 /* Set to '1' for more logging of test cases */
+#define VERBOSE 1 /* Set to '1' for more logging of test cases */

That seems to be unrelated.

--
Stefan

Quoted 3 lines
arch/arm/probes/kprobes/test-core.h +0 -0
 #ifdef CONFIG_THUMB2_KERNEL
 #define NORMAL_ISA "16"
#3 lore raw

RFC PATCH 1/1 arm: Fix cache inconsistency when using fixmap

Quoted 1 line · Ard Biesheuvel

On 25 February 2017 at 01:06, Stefan Agner <[email protected]> wrote:

[...]

Quoted 63 lines
Quoted 42 lines

Early fixmap gets shut down before going SMP, so non shared mappings are not an issue there. But FIXMAP_PAGE_NORMAL is also used for regular fixmap, which is essentially the problem. Also, regular fixmap does not make sure of the cache policy due to that, but just uses the hardcoded writeback policy...

With that in mind, populating kern_pgprot with a reasonable default which works for early fixmap might be a reasonable approach. Later kern_pgprot gets setup with the appropriate shared attribute and cache policy...

arch/arm/include/asm/fixmap.h +0 -0
index 5c17d2dec777..c663e562716c 100644
@@ -39,12 +39,11 @@ static const enum fixed_addresses

__end_of_fixed_addresses =
__end_of_fixmap_region > __end_of_early_ioremap_region ? __end_of_fixmap_region : __end_of_early_ioremap_region;

-#define FIXMAP_PAGE_COMMON (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN
| L_PTE_DIRTY)

arch/arm/include/asm/fixmap.h +0 -2
-
-#define FIXMAP_PAGE_NORMAL     (FIXMAP_PAGE_COMMON |

L_PTE_MT_WRITEBACK)

arch/arm/include/asm/fixmap.h +2 -0
+#define FIXMAP_PAGE_NORMAL     (pgprot_kernel | L_PTE_XN)
 #define FIXMAP_PAGE_RO         (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
+#define FIXMAP_PAGE_COMMON     (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN

| L_PTE_DIRTY)
#define FIXMAP_PAGE_IO (FIXMAP_PAGE_COMMON |
L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

arch/arm/mm/mmu.c +0 -0
index 4001dd15818d..9cb7c559263f 100644

arch/arm/mm/mmu.c pmdval_t user_pmd_table = _PAGE_USER_TABLE; · line 72

Quoted 9 lines
arch/arm/mm/mmu.c +2 -1
@@ -65,7 +65,8 @@ pmdval_t user_pmd_table = _PAGE_USER_TABLE;
 static unsigned int cachepolicy __initdata = CPOLICY_WRITEBACK;
 static unsigned int ecc_mask __initdata = 0;
 pgprot_t pgprot_user;
-pgprot_t pgprot_kernel;
+pgprot_t pgprot_kernel = __pgprot(L_PTE_PRESENT | L_PTE_YOUNG |
+                                 L_PTE_DIRTY | L_PTE_MT_WRITEBACK);
 pgprot_t pgprot_hyp_device;
 pgprot_t pgprot_s2;
 pgprot_t pgprot_s2_device;

This looks correct to me, and avoids using shareable attributes inadvertently.

However, we could make this a bit more private to the fixmap code by using something like

arch/arm/include/asm/fixmap.h +0 -0
index 5c17d2dec777..6c375aea8f22 100644

arch/arm/include/asm/fixmap.h static const enum fixed_addresses __end_of_fixed_addresses = · line 47

Quoted 9 lines
arch/arm/include/asm/fixmap.h +2 -2
@@ -41,11 +41,14 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON     (L_PTE_YOUNG | L_PTE_PRESENT |
 L_PTE_XN | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL     (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO         (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define __FIXMAP_MT_NORMAL     ((pgprot_val(pgprot_kernel) ?:
 FIXMAP_PAGE_COMMON) | \
+                                L_PTE_MT_WRITEBACK)

... only this L_PTE_MT_WRITEBACK belongs next to FIXMAP_PAGE_COMMON, since pgprot_kernel will already have a memory type index set, which may conflict with MT_WRITEBACK

Quoted 66 lines
arch/arm/include/asm/fixmap.h +3 -1
+#define FIXMAP_PAGE_NORMAL     __pgprot(__FIXMAP_MT_NORMAL)
+#define FIXMAP_PAGE_RO         __pgprot(__FIXMAP_MT_NORMAL | L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO         (FIXMAP_PAGE_COMMON |
 L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
+#define FIXMAP_PAGE_IO         __pgprot(FIXMAP_PAGE_COMMON |

L_PTE_MT_DEV_SHARED | \
+ L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

#define __early_set_fixmap __set_fixmap

(and drop the change to mm/mmu.c), which boils down to the same for fixmap but does not affect other users of pgprot_kernel. Also, it appears these definitions are broken under STRICT_MM_TYPECHECKS so this is a good opportunity to get that fixed as well.

Thank,
Ard.

Quoted 44 lines
Quoted 30 lines

That's why I'm marking this as an RFC, perhaps the correct fix is to rework the code to use the defines from arch/arm/include/asm/pgtable.h

 arch/arm/include/asm/fixmap.h       | 2 +-
 arch/arm/probes/kprobes/test-core.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/arm/include/asm/fixmap.h +1 -1
index 5c17d2dec777..26d4a4677cd7 100644
@@ -41,7 +41,7 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON   (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN |
 L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL   (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL   (FIXMAP_PAGE_COMMON | L_PTE_SHARED |
 L_PTE_MT_WRITEBACK)
 #define FIXMAP_PAGE_RO               (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)

/* Used by set_fixmap_(io|nocache), both meant for mapping a device */

arch/arm/probes/kprobes/test-core.h +0 -0
index 94285203e9f7..cde2b4e9358a 100644

arch/arm/probes/kprobes/test-core.h line 11

Quoted 5 lines
arch/arm/probes/kprobes/test-core.h +1 -1
@@ -8,7 +8,7 @@
  * published by the Free Software Foundation.
  */
 
-#define VERBOSE 0 /* Set to '1' for more logging of test cases */
+#define VERBOSE 1 /* Set to '1' for more logging of test cases */

That seems to be unrelated.

--
Stefan

Quoted 3 lines
arch/arm/probes/kprobes/test-core.h +0 -0
 #ifdef CONFIG_THUMB2_KERNEL
 #define NORMAL_ISA "16"
#4 lore raw

RFC PATCH 1/1 arm: Fix cache inconsistency when using fixmap

In reply to Ard Biesheuvel
Quoted 9 lines · Ard Biesheuvel

However, we could make this a bit more private to the fixmap code by using something like

arch/arm/include/asm/fixmap.h +0 -0
index 5c17d2dec777..6c375aea8f22 100644

arch/arm/include/asm/fixmap.h static const enum fixed_addresses __end_of_fixed_addresses = · line 45

Quoted 7 lines
arch/arm/include/asm/fixmap.h +1 -2
@@ -41,11 +41,14 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON     (L_PTE_YOUNG | L_PTE_PRESENT |
 L_PTE_XN | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL     (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO         (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define __FIXMAP_MT_NORMAL     ((pgprot_val(pgprot_kernel) ?:

The code above misses out setting L_PTE_XN when using pgprot_kernel

Quoted 13 lines
arch/arm/include/asm/fixmap.h +4 -1
 FIXMAP_PAGE_COMMON) | \
+                                L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL     __pgprot(__FIXMAP_MT_NORMAL)
+#define FIXMAP_PAGE_RO         __pgprot(__FIXMAP_MT_NORMAL | L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO         (FIXMAP_PAGE_COMMON |
 L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
+#define FIXMAP_PAGE_IO         __pgprot(FIXMAP_PAGE_COMMON |

L_PTE_MT_DEV_SHARED | \
+ L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

Quoted 6 lines

#define __early_set_fixmap __set_fixmap

(and drop the change to mm/mmu.c), which boils down to the same for fixmap but does not affect other users of pgprot_kernel. Also, it appears these definitions are broken under STRICT_MM_TYPECHECKS so this is a good opportunity to get that fixed as well.

I like this method better because as you say it keeps things private to fixmap, and it doesn't risk affecting other things.

As for getting STRICT_MM_TYPECHECKS working that looks good too, but should be a separate cleanup patch, especially as a fix for the cache problem possibly should go to stable kernels.

I also think FIXMAP_PAGE_COMMON should be renamed (prefixed with '__') as it's an implementation detail an not a memory type used with fixmap.

So I was thinking, one patch as a bugfix:

----------------------------------------------------------------------

arch/arm/include/asm/fixmap.h +2 -1
index 5c17d2dec777..4e6784dc5668 100644
@@ -41,7 +41,8 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON	(L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL	(FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL	(pgprot_kernel ? pgprot_kernel | L_PTE_XN : \
+					FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
 #define FIXMAP_PAGE_RO		(FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */

----------------------------------------------------------------------

Second patch as a cleanup. Note this is different to Ard's version as it uses PAGE_KERNEL rather than open coding the same code from pgtable.h to add XN permisions (Also, the default generic definition of FIXMAP_PAGE_NORMAL is PAGE_KERNEL, and the only reason we want to change it is to select an alternate value if that is not yet setup)

----------------------------------------------------------------------

arch/arm/include/asm/fixmap.h +6 -5
index 4e6784dc5668..ec689c6aa644 100644
@@ -39,14 +39,15 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 	__end_of_fixmap_region > __end_of_early_ioremap_region ?
 	__end_of_fixmap_region : __end_of_early_ioremap_region;
 
-#define FIXMAP_PAGE_COMMON	(L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN | L_PTE_DIRTY)
+#define __FIXMAP_PAGE_COMMON	(L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL	(pgprot_kernel ? pgprot_kernel | L_PTE_XN : \
-					FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO		(FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define FIXMAP_PAGE_NORMAL	(pgprot_val(pgprot_kernel) ? PAGE_KERNEL : \
+				 __pgprot(__FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK))
+#define FIXMAP_PAGE_RO		_MOD_PROT(FIXMAP_PAGE_NORMAL, L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO		(FIXMAP_PAGE_COMMON | L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
+#define FIXMAP_PAGE_IO         __pgprot(__FIXMAP_PAGE_COMMON | L_PTE_MT_DEV_SHARED | \
+                                        L_PTE_SHARED)
 #define FIXMAP_PAGE_NOCACHE	FIXMAP_PAGE_IO
 
 #define __early_set_fixmap	__set_fixmap

----------------------------------------------------------------------

#5 lore raw

RFC PATCH 1/1 arm: Fix cache inconsistency when using fixmap

Quoted 36 lines · Jon Medhurst (Tixy)
Quoted 9 lines · Ard Biesheuvel

However, we could make this a bit more private to the fixmap code by using something like

arch/arm/include/asm/fixmap.h +0 -0
index 5c17d2dec777..6c375aea8f22 100644

arch/arm/include/asm/fixmap.h static const enum fixed_addresses __end_of_fixed_addresses = · line 45

Quoted 7 lines
arch/arm/include/asm/fixmap.h +1 -2
@@ -41,11 +41,14 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON     (L_PTE_YOUNG | L_PTE_PRESENT |
 L_PTE_XN | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL     (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO         (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define __FIXMAP_MT_NORMAL     ((pgprot_val(pgprot_kernel) ?:

The code above misses out setting L_PTE_XN when using pgprot_kernel

Quoted 15 lines
arch/arm/include/asm/fixmap.h +4 -1
 FIXMAP_PAGE_COMMON) | \
+                                L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL     __pgprot(__FIXMAP_MT_NORMAL)
+#define FIXMAP_PAGE_RO         __pgprot(__FIXMAP_MT_NORMAL | L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO         (FIXMAP_PAGE_COMMON |
 L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
+#define FIXMAP_PAGE_IO         __pgprot(FIXMAP_PAGE_COMMON |

L_PTE_MT_DEV_SHARED | \
+ L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

#define __early_set_fixmap __set_fixmap

Quoted 7 lines
Quoted 4 lines

(and drop the change to mm/mmu.c), which boils down to the same for fixmap but does not affect other users of pgprot_kernel. Also, it appears these definitions are broken under STRICT_MM_TYPECHECKS so this is a good opportunity to get that fixed as well.

I like this method better because as you say it keeps things private to fixmap, and it doesn't risk affecting other things.

Yes, I agree this is nicer than my proposal.

Quoted 63 lines

As for getting STRICT_MM_TYPECHECKS working that looks good too, but should be a separate cleanup patch, especially as a fix for the cache problem possibly should go to stable kernels.

I also think FIXMAP_PAGE_COMMON should be renamed (prefixed with '__') as it's an implementation detail an not a memory type used with fixmap.

So I was thinking, one patch as a bugfix:

----------------------------------------------------------------------

arch/arm/include/asm/fixmap.h +2 -1
index 5c17d2dec777..4e6784dc5668 100644
@@ -41,7 +41,8 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON	(L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN |
 L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL	(FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL	(pgprot_kernel ? pgprot_kernel | L_PTE_XN : \
+					FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
 #define FIXMAP_PAGE_RO		(FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
 

/* Used by set_fixmap_(io|nocache), both meant for mapping a device */
----------------------------------------------------------------------

Second patch as a cleanup. Note this is different to Ard's version as it uses PAGE_KERNEL rather than open coding the same code from pgtable.h to add XN permisions (Also, the default generic definition of FIXMAP_PAGE_NORMAL is PAGE_KERNEL, and the only reason we want to change it is to select an alternate value if that is not yet setup)

----------------------------------------------------------------------

arch/arm/include/asm/fixmap.h +5 -5
index 4e6784dc5668..ec689c6aa644 100644
@@ -39,14 +39,15 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 	__end_of_fixmap_region > __end_of_early_ioremap_region ?
 	__end_of_fixmap_region : __end_of_early_ioremap_region;
 
-#define FIXMAP_PAGE_COMMON	(L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN |
 L_PTE_DIRTY)
+#define __FIXMAP_PAGE_COMMON	(L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN
 | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL	(pgprot_kernel ? pgprot_kernel | L_PTE_XN : \
-					FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO		(FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define FIXMAP_PAGE_NORMAL	(pgprot_val(pgprot_kernel) ? PAGE_KERNEL : \
+				 __pgprot(__FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK))
+#define FIXMAP_PAGE_RO		_MOD_PROT(FIXMAP_PAGE_NORMAL, L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO		(FIXMAP_PAGE_COMMON | L_PTE_MT_DEV_SHARED |
 L_PTE_SHARED)
+#define FIXMAP_PAGE_IO         __pgprot(__FIXMAP_PAGE_COMMON |

L_PTE_MT_DEV_SHARED | \
+ L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

Quoted 2 lines

#define __early_set_fixmap __set_fixmap
----------------------------------------------------------------------

Fix and cleanup looks good to me,

--
Stefan

#6 lore raw

RFC PATCH 1/1 arm: Fix cache inconsistency when using fixmap

Quoted 110 lines · Stefan Agner
Quoted 36 lines · Jon Medhurst (Tixy)
Quoted 9 lines · Ard Biesheuvel

However, we could make this a bit more private to the fixmap code by using something like

arch/arm/include/asm/fixmap.h +0 -0
index 5c17d2dec777..6c375aea8f22 100644

arch/arm/include/asm/fixmap.h static const enum fixed_addresses __end_of_fixed_addresses = · line 45

Quoted 7 lines
arch/arm/include/asm/fixmap.h +1 -2
@@ -41,11 +41,14 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON     (L_PTE_YOUNG | L_PTE_PRESENT |
 L_PTE_XN | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL     (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO         (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define __FIXMAP_MT_NORMAL     ((pgprot_val(pgprot_kernel) ?:

The code above misses out setting L_PTE_XN when using pgprot_kernel

Quoted 15 lines
arch/arm/include/asm/fixmap.h +4 -1
 FIXMAP_PAGE_COMMON) | \
+                                L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL     __pgprot(__FIXMAP_MT_NORMAL)
+#define FIXMAP_PAGE_RO         __pgprot(__FIXMAP_MT_NORMAL | L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO         (FIXMAP_PAGE_COMMON |
 L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
+#define FIXMAP_PAGE_IO         __pgprot(FIXMAP_PAGE_COMMON |

L_PTE_MT_DEV_SHARED | \
+ L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

#define __early_set_fixmap __set_fixmap

Quoted 7 lines
Quoted 4 lines

(and drop the change to mm/mmu.c), which boils down to the same for fixmap but does not affect other users of pgprot_kernel. Also, it appears these definitions are broken under STRICT_MM_TYPECHECKS so this is a good opportunity to get that fixed as well.

I like this method better because as you say it keeps things private to fixmap, and it doesn't risk affecting other things.

Yes, I agree this is nicer than my proposal.

Quoted 63 lines

As for getting STRICT_MM_TYPECHECKS working that looks good too, but should be a separate cleanup patch, especially as a fix for the cache problem possibly should go to stable kernels.

I also think FIXMAP_PAGE_COMMON should be renamed (prefixed with '__') as it's an implementation detail an not a memory type used with fixmap.

So I was thinking, one patch as a bugfix:

----------------------------------------------------------------------

arch/arm/include/asm/fixmap.h +2 -1
index 5c17d2dec777..4e6784dc5668 100644
@@ -41,7 +41,8 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON   (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN |
 L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL   (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL   (pgprot_kernel ? pgprot_kernel | L_PTE_XN : \
+                                     FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
 #define FIXMAP_PAGE_RO               (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
 

/* Used by set_fixmap_(io|nocache), both meant for mapping a device */
----------------------------------------------------------------------

Second patch as a cleanup. Note this is different to Ard's version as it uses PAGE_KERNEL rather than open coding the same code from pgtable.h to add XN permisions (Also, the default generic definition of FIXMAP_PAGE_NORMAL is PAGE_KERNEL, and the only reason we want to change it is to select an alternate value if that is not yet setup)

----------------------------------------------------------------------

arch/arm/include/asm/fixmap.h +5 -5
index 4e6784dc5668..ec689c6aa644 100644
@@ -39,14 +39,15 @@ static const enum fixed_addresses __end_of_fixed_addresses =
      __end_of_fixmap_region > __end_of_early_ioremap_region ?
      __end_of_fixmap_region : __end_of_early_ioremap_region;
 
-#define FIXMAP_PAGE_COMMON   (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN |
 L_PTE_DIRTY)
+#define __FIXMAP_PAGE_COMMON (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN
 | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL   (pgprot_kernel ? pgprot_kernel | L_PTE_XN : \
-                                     FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO               (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define FIXMAP_PAGE_NORMAL   (pgprot_val(pgprot_kernel) ? PAGE_KERNEL : \
+                              __pgprot(__FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK))
+#define FIXMAP_PAGE_RO               _MOD_PROT(FIXMAP_PAGE_NORMAL, L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO               (FIXMAP_PAGE_COMMON | L_PTE_MT_DEV_SHARED |
 L_PTE_SHARED)
+#define FIXMAP_PAGE_IO         __pgprot(__FIXMAP_PAGE_COMMON |

L_PTE_MT_DEV_SHARED | \
+ L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

Quoted 7 lines
Quoted 2 lines

#define __early_set_fixmap __set_fixmap
----------------------------------------------------------------------

Fix and cleanup looks good to me,

Likewise,

#7 lore raw

RFC PATCH 1/1 arm: Fix cache inconsistency when using fixmap

Quoted 116 lines · Ard Biesheuvel
Quoted 115 lines · Stefan Agner
Quoted 36 lines · Jon Medhurst (Tixy)
Quoted 9 lines · Ard Biesheuvel

However, we could make this a bit more private to the fixmap code by using something like

arch/arm/include/asm/fixmap.h +0 -0
index 5c17d2dec777..6c375aea8f22 100644

arch/arm/include/asm/fixmap.h static const enum fixed_addresses __end_of_fixed_addresses = · line 45

Quoted 7 lines
arch/arm/include/asm/fixmap.h +1 -2
@@ -41,11 +41,14 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON     (L_PTE_YOUNG | L_PTE_PRESENT |
 L_PTE_XN | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL     (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO         (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define __FIXMAP_MT_NORMAL     ((pgprot_val(pgprot_kernel) ?:

The code above misses out setting L_PTE_XN when using pgprot_kernel

Quoted 15 lines
arch/arm/include/asm/fixmap.h +4 -1
 FIXMAP_PAGE_COMMON) | \
+                                L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL     __pgprot(__FIXMAP_MT_NORMAL)
+#define FIXMAP_PAGE_RO         __pgprot(__FIXMAP_MT_NORMAL | L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO         (FIXMAP_PAGE_COMMON |
 L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
+#define FIXMAP_PAGE_IO         __pgprot(FIXMAP_PAGE_COMMON |

L_PTE_MT_DEV_SHARED | \
+ L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

#define __early_set_fixmap __set_fixmap

Quoted 7 lines
Quoted 4 lines

(and drop the change to mm/mmu.c), which boils down to the same for fixmap but does not affect other users of pgprot_kernel. Also, it appears these definitions are broken under STRICT_MM_TYPECHECKS so this is a good opportunity to get that fixed as well.

I like this method better because as you say it keeps things private to fixmap, and it doesn't risk affecting other things.

Yes, I agree this is nicer than my proposal.

Quoted 63 lines

As for getting STRICT_MM_TYPECHECKS working that looks good too, but should be a separate cleanup patch, especially as a fix for the cache problem possibly should go to stable kernels.

I also think FIXMAP_PAGE_COMMON should be renamed (prefixed with '__') as it's an implementation detail an not a memory type used with fixmap.

So I was thinking, one patch as a bugfix:

----------------------------------------------------------------------

arch/arm/include/asm/fixmap.h +2 -1
index 5c17d2dec777..4e6784dc5668 100644
@@ -41,7 +41,8 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON   (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN |
 L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL   (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL   (pgprot_kernel ? pgprot_kernel | L_PTE_XN : \
+                                     FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
 #define FIXMAP_PAGE_RO               (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
 

/* Used by set_fixmap_(io|nocache), both meant for mapping a device */
----------------------------------------------------------------------

Second patch as a cleanup. Note this is different to Ard's version as it uses PAGE_KERNEL rather than open coding the same code from pgtable.h to add XN permisions (Also, the default generic definition of FIXMAP_PAGE_NORMAL is PAGE_KERNEL, and the only reason we want to change it is to select an alternate value if that is not yet setup)

----------------------------------------------------------------------

arch/arm/include/asm/fixmap.h +5 -5
index 4e6784dc5668..ec689c6aa644 100644
@@ -39,14 +39,15 @@ static const enum fixed_addresses __end_of_fixed_addresses =
      __end_of_fixmap_region > __end_of_early_ioremap_region ?
      __end_of_fixmap_region : __end_of_early_ioremap_region;
 
-#define FIXMAP_PAGE_COMMON   (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN |
 L_PTE_DIRTY)
+#define __FIXMAP_PAGE_COMMON (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN
 > L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL   (pgprot_kernel ? pgprot_kernel | L_PTE_XN : \
-                                     FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO               (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define FIXMAP_PAGE_NORMAL   (pgprot_val(pgprot_kernel) ? PAGE_KERNEL : \
+                              __pgprot(__FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK))
+#define FIXMAP_PAGE_RO               _MOD_PROT(FIXMAP_PAGE_NORMAL, L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO               (FIXMAP_PAGE_COMMON | L_PTE_MT_DEV_SHARED |
 L_PTE_SHARED)
+#define FIXMAP_PAGE_IO         __pgprot(__FIXMAP_PAGE_COMMON |

L_PTE_MT_DEV_SHARED | \
+ L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

Quoted 2 lines

#define __early_set_fixmap __set_fixmap
----------------------------------------------------------------------

Fix and cleanup looks good to me,

Quoted 6 lines
Quoted 2 lines

Likewise,

Thanks both. I'll do some more testing then post proper patches with commit messages, and add something like

Based on changes

#8 lore raw

RFC PATCH 1/1 arm: Fix cache inconsistency when using fixmap

Quoted 122 lines · Jon Medhurst (Tixy)
Quoted 121 lines · Ard Biesheuvel
Quoted 110 lines · Stefan Agner
Quoted 36 lines · Jon Medhurst (Tixy)
Quoted 9 lines · Ard Biesheuvel

However, we could make this a bit more private to the fixmap code by using something like

arch/arm/include/asm/fixmap.h +0 -0
index 5c17d2dec777..6c375aea8f22 100644

arch/arm/include/asm/fixmap.h static const enum fixed_addresses __end_of_fixed_addresses = · line 45

Quoted 7 lines
arch/arm/include/asm/fixmap.h +1 -2
@@ -41,11 +41,14 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON     (L_PTE_YOUNG | L_PTE_PRESENT |
 L_PTE_XN | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL     (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO         (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define __FIXMAP_MT_NORMAL     ((pgprot_val(pgprot_kernel) ?:

The code above misses out setting L_PTE_XN when using pgprot_kernel

Quoted 15 lines
arch/arm/include/asm/fixmap.h +4 -1
 FIXMAP_PAGE_COMMON) | \
+                                L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL     __pgprot(__FIXMAP_MT_NORMAL)
+#define FIXMAP_PAGE_RO         __pgprot(__FIXMAP_MT_NORMAL | L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO         (FIXMAP_PAGE_COMMON |
 L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
+#define FIXMAP_PAGE_IO         __pgprot(FIXMAP_PAGE_COMMON |

L_PTE_MT_DEV_SHARED | \
+ L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

#define __early_set_fixmap __set_fixmap

Quoted 7 lines
Quoted 4 lines

(and drop the change to mm/mmu.c), which boils down to the same for fixmap but does not affect other users of pgprot_kernel. Also, it appears these definitions are broken under STRICT_MM_TYPECHECKS so this is a good opportunity to get that fixed as well.

I like this method better because as you say it keeps things private to fixmap, and it doesn't risk affecting other things.

Yes, I agree this is nicer than my proposal.

Quoted 63 lines

As for getting STRICT_MM_TYPECHECKS working that looks good too, but should be a separate cleanup patch, especially as a fix for the cache problem possibly should go to stable kernels.

I also think FIXMAP_PAGE_COMMON should be renamed (prefixed with '__') as it's an implementation detail an not a memory type used with fixmap.

So I was thinking, one patch as a bugfix:

----------------------------------------------------------------------

arch/arm/include/asm/fixmap.h +2 -1
index 5c17d2dec777..4e6784dc5668 100644
@@ -41,7 +41,8 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON   (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN |
 L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL   (FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
+#define FIXMAP_PAGE_NORMAL   (pgprot_kernel ? pgprot_kernel | L_PTE_XN : \
+                                     FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
 #define FIXMAP_PAGE_RO               (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
 

/* Used by set_fixmap_(io|nocache), both meant for mapping a device */
----------------------------------------------------------------------

Second patch as a cleanup. Note this is different to Ard's version as it uses PAGE_KERNEL rather than open coding the same code from pgtable.h to add XN permisions (Also, the default generic definition of FIXMAP_PAGE_NORMAL is PAGE_KERNEL, and the only reason we want to change it is to select an alternate value if that is not yet setup)

----------------------------------------------------------------------

arch/arm/include/asm/fixmap.h +5 -5
index 4e6784dc5668..ec689c6aa644 100644
@@ -39,14 +39,15 @@ static const enum fixed_addresses __end_of_fixed_addresses =
      __end_of_fixmap_region > __end_of_early_ioremap_region ?
      __end_of_fixmap_region : __end_of_early_ioremap_region;
 
-#define FIXMAP_PAGE_COMMON   (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN |
 L_PTE_DIRTY)
+#define __FIXMAP_PAGE_COMMON (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN
 > L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL   (pgprot_kernel ? pgprot_kernel | L_PTE_XN : \
-                                     FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK)
-#define FIXMAP_PAGE_RO               (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define FIXMAP_PAGE_NORMAL   (pgprot_val(pgprot_kernel) ? PAGE_KERNEL : \
+                              __pgprot(__FIXMAP_PAGE_COMMON | L_PTE_MT_WRITEBACK))
+#define FIXMAP_PAGE_RO               _MOD_PROT(FIXMAP_PAGE_NORMAL, L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO               (FIXMAP_PAGE_COMMON | L_PTE_MT_DEV_SHARED |
 L_PTE_SHARED)
+#define FIXMAP_PAGE_IO         __pgprot(__FIXMAP_PAGE_COMMON |

L_PTE_MT_DEV_SHARED | \
+ L_PTE_SHARED)
#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO

Quoted 7 lines
Quoted 2 lines

#define __early_set_fixmap __set_fixmap
----------------------------------------------------------------------

Fix and cleanup looks good to me,

Likewise,

Quoted 8 lines
Quoted 1 line

Thanks both. I'll do some more testing then post proper patches with commit messages, and add something like

Based on changes

If you insist, but between Stefan, you and me, I am hardly the person to have given the most substantial input into this discussion