arm: support get_user_pages_fast
Subsystems (2)
v2 cover arm: support get_user_pages_fast
get_user_pages_fast helps app's launching time because it could reduce mmap_sem lock contentions when app is launching.
Minchan posted a RFC patch and patch v1 [1] in 2018 and currently he has no bandwidth to rework this patch, so I rebase Minchan's patch (arm: support get_user_pages_fast) and modify the patch according to Catalin's comment and re-post this patch.
First, to use generic get_user_pages_fast code, use strict mm type in pgtable-2level-types.
2nd patch: reorder memory type table to use 5th bit of the page table
3rd patch: introduces L_PTE_SPECIAL for arm so that last patch can support get_user_pags_fast.
4th patch: remove L_PTE_MT_VECTORS and use read-only type.
[1] https://lore.kernel.org/patchwork/cover/986573/
Change since v1:
* modify comment
* remove L_PTE_MT_VECTORS and use read-only type
Change since RFC:
* Remove CPU_v7M - Catalin
* define DEV_NONSHARED to SHARED - Catalin
* https://lkml.kernel.org/r/[email protected]
Miles Chen (2):
arm: mm: use strict p[gum]d types
arm: replace vector mem type with read-only type
Minchan Kim (2):
arm: mm: reordering memory type table
arm: mm: introduce L_PTE_SPECIAL
arch/arm/Kconfig | 4 ++-- arch/arm/include/asm/fixmap.h | 6 ++--- arch/arm/include/asm/pgtable-2level-types.h | 26 --------------------- arch/arm/include/asm/pgtable-2level.h | 22 +++++++++++++---- arch/arm/include/asm/pgtable-3level.h | 6 ----- arch/arm/include/asm/pgtable.h | 13 +++++++++++ arch/arm/mm/mmu.c | 7 +++--- arch/arm/mm/proc-macros.S | 2 +- 8 files changed, 40 insertions(+), 46 deletions(-)
PATCH v2 1/4 arm: mm: use strict p[gum]d types
When use mm/gup.c, it is necessary to use strict p*d types. Otherwise, we will get build breaks such as:
mm/gup.c:2589:9: error: incompatible pointer types passing 'pmdval_t **' (aka 'unsigned int **') to parameter of type 'pgd_t *'
(aka 'pmdval_t (*)[2]') [-Werror,-Wincompatible-pointer-types] mm/gup.c:2616:9: error: array initializer must be an initializer list
Fix some types errors in fixmap.h after using strict types.
--- arch/arm/include/asm/fixmap.h | 6 ++--- arch/arm/include/asm/pgtable-2level-types.h | 26 --------------------- arch/arm/include/asm/pgtable-2level.h | 1 + 3 files changed, 4 insertions(+), 29 deletions(-)
arch/arm/include/asm/fixmap.h +3 -3
@@ -42,11 +42,11 @@ static const enum fixed_addresses __end_of_fixed_addresses =#define FIXMAP_PAGE_COMMON (L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN | L_PTE_DIRTY) -#define FIXMAP_PAGE_NORMAL (pgprot_kernel | L_PTE_XN) -#define FIXMAP_PAGE_RO (FIXMAP_PAGE_NORMAL | L_PTE_RDONLY) +#define FIXMAP_PAGE_NORMAL __pgprot(pgprot_val(pgprot_kernel) | L_PTE_XN) +#define FIXMAP_PAGE_RO __pgprot(pgprot_val(FIXMAP_PAGE_NORMAL) | L_PTE_RDONLY)/* Used by set_fixmap_(io|nocache), both meant for mapping a device */ -#define FIXMAP_PAGE_IO (FIXMAP_PAGE_COMMON | L_PTE_MT_DEV_SHARED | L_PTE_SHARED) +#define FIXMAP_PAGE_IO __pgprot(FIXMAP_PAGE_COMMON | L_PTE_MT_DEV_SHARED | L_PTE_SHARED)#define FIXMAP_PAGE_NOCACHE FIXMAP_PAGE_IO#define __early_set_fixmap __set_fixmap
arch/arm/include/asm/pgtable-2level-types.h +0 -26
@@ -12,12 +12,6 @@typedef u32 pteval_t ;typedef u32 pmdval_t ; -#undef STRICT_MM_TYPECHECKS - -#ifdef STRICT_MM_TYPECHECKS -/* -* These are used to make use of C type-checking.. -*/ typedef struct {pteval_t pte; }pte_t ;typedef struct {pmdval_t pmd; }pmd_t ;typedef struct {pmdval_t pgd[2 ]; }pgd_t ;
@@ -32,24 +26,4 @@ typedef struct { pteval_t pgprot; } pgprot_t;#define __pmd (x) ((pmd_t) { (x) } )#define __pgprot (x) ((pgprot_t) { (x) } ) -#else -/* -* .. while these make it easier on the compiler -*/ -typedef pteval_t pte_t ; -typedef pmdval_t pmd_t ; -typedef pmdval_t pgd_t [2 ]; -typedef pteval_t pgprot_t ; - -#define pte_val (x) (x) -#define pmd_val (x) (x) -#define pgd_val (x) ((x)[0]) -#define pgprot_val (x) (x) - -#define __pte (x) (x) -#define __pmd (x) (x) -#define __pgprot (x) (x) - -#endif /* STRICT_MM_TYPECHECKS */ -#endif /* _ASM_PGTABLE_2LEVEL_TYPES_H */
arch/arm/include/asm/pgtable-2level.h +1 -0
@@ -180,6 +180,7 @@#define pud_none (pud) (0)#define pud_bad (pud) (0)#define pud_present (pud) (1) +#define pud_page (pud) pmd_page(__pmd(pud_val(pud)))#define pud_clear (pudp) do { } while (0)#define set_pud (pud,pudp) do { } while (0)
Signature · 1 line
2.18.0
PATCH v2 2/4 arm: mm: reordering memory type table
In reply to Miles ChenFrom: Minchan Kim <[email protected]>
To use bit 5 in page table as L_PTE_SPECIAL, we need a room for that. It seems we don't need 4 bits for the memory type with ARMv6+. If it's true, let's reorder bits to make bit 5 free.
We will use the bit for L_PTE_SPECIAL in next patch.
A note from Catalin in [1]:
"
Quoted 6 lines
Anyway, on ARMv7 or ARMv6+LPAE, the non-shared device gets mapped to shared device in hardware. Looking through the arm32 code, it seems that
MT_DEVICE_NONSHAREDis used byarch/arm/mach-shmobile/setup-r8a7779.cand IIUC that's a v7 platform (R-Car H1, Cortex-A9). I think the above should be defined toL_PTE_MT_DEV_SHARED, unless I miss any place whereDEV_NONSHAREDis relevant on ARMv6 (adding Simon to confirm on shmbile).
"
[1] https://lore.kernel.org/patchwork/patch/986574/
--- arch/arm/include/asm/pgtable-2level.h | 21 +++++++++++++++++---- arch/arm/mm/proc-macros.S | 4 ++-- 2 files changed, 19 insertions(+), 6 deletions(-)
arch/arm/include/asm/pgtable-2level.h +17 -4
@@ -161,14 +161,27 @@#define L_PTE_MT_BUFFERABLE (_AT(pteval_t, 0x01) << 2)/* 0001 */ #define L_PTE_MT_WRITETHROUGH (_AT(pteval_t, 0x02) << 2)/* 0010 */ #define L_PTE_MT_WRITEBACK (_AT(pteval_t, 0x03) << 2)/* 0011 */ +#define L_PTE_MT_DEV_SHARED (_AT(pteval_t, 0x04) << 2)/* 0100 */ +#define L_PTE_MT_VECTORS (_AT(pteval_t, 0x05) << 2)/* 0101 */ #define L_PTE_MT_MINICACHE (_AT(pteval_t, 0x06) << 2)/* 0110 (sa1100, xscale) */ #define L_PTE_MT_WRITEALLOC (_AT(pteval_t, 0x07) << 2)/* 0111 */ -#define L_PTE_MT_DEV_SHARED (_AT(pteval_t, 0x04) << 2)/* 0100 */ -#define L_PTE_MT_DEV_NONSHARED (_AT(pteval_t, 0x0c) << 2)/* 1100 */ +#if defined(CONFIG_CPU_V7 ) || defined(CONFIG_CPU_V6 ) || defined(CONFIG_CPU_V6K ) +/* +* On ARMv7 or ARMv7+LPAE, the non-shared and shared device types get +* mapped to the same TEX remapping index. On classic ARMv7, the +* shareability is controlled by the PRRR[17:16] field, indexed by +* L_PTE_SHARED. On ARMv7+LPAE the device mapping is always shareable. +*/ +#define L_PTE_MT_DEV_NONSHARED L_PTE_MT_DEV_SHARED +#define L_PTE_MT_DEV_WC L_PTE_MT_BUFFERABLE +#define L_PTE_MT_DEV_CACHED L_PTE_MT_WRITEBACK +#define L_PTE_MT_MASK (_AT(pteval_t, 0x07) << 2) +#else #define L_PTE_MT_DEV_WC (_AT(pteval_t, 0x09) << 2)/* 1001 */ #define L_PTE_MT_DEV_CACHED (_AT(pteval_t, 0x0b) << 2)/* 1011 */ -#define L_PTE_MT_VECTORS (_AT(pteval_t, 0x0f) << 2)/* 1111 */ -#define L_PTE_MT_MASK (_AT(pteval_t, 0x0f) << 2) +#define L_PTE_MT_DEV_NONSHARED (_AT(pteval_t, 0x0c) << 2)/* 1100 */ +#define L_PTE_MT_MASK (_AT(pteval_t, 0x0f) << 2) +#endif #ifndef __ASSEMBLY__
arch/arm/mm/proc-macros.S +2 -2
@@ -137,7 +137,7 @@ .long PTE_CACHEABLE @ L_PTE_MT_WRITETHROUGH .long PTE_CACHEABLE | PTE_BUFFERABLE @ L_PTE_MT_WRITEBACK .long PTE_BUFFERABLE @ L_PTE_MT_DEV_SHARED - .long 0x00 @ unused + .long PTE_CACHEABLE | PTE_BUFFERABLE | PTE_EXT_APX @ L_PTE_MT_VECTORS .long 0x00 @ L_PTE_MT_MINICACHE (not present) .long PTE_EXT_TEX(1) | PTE_CACHEABLE | PTE_BUFFERABLE @ L_PTE_MT_WRITEALLOC .long 0x00 @ unused
@@ -147,7 +147,7 @@ .long PTE_EXT_TEX(2) @ L_PTE_MT_DEV_NONSHARED .long 0x00 @ unused .long 0x00 @ unused - .long PTE_CACHEABLE | PTE_BUFFERABLE | PTE_EXT_APX @ L_PTE_MT_VECTORS + .long 0x00 @ unused .endm .macro armv6_set_pte_ext pfx
Signature · 1 line
2.18.0
Russell King - ARM Linux admin
maintainerRe: [PATCH v2 2/4] arm: mm: reordering memory type table
Quoted 39 lines · Miles Chen
From: Minchan Kim <[email protected]>
To use bit 5 in page table as
L_PTE_SPECIAL, we need a room for that. It seems we don't need 4 bits for the memory type with ARMv6+. If it's true, let's reorder bits to make bit 5 free.We will use the bit for
L_PTE_SPECIALin next patch.A note from Catalin in [1]:
"Quoted 6 lines
Anyway, on ARMv7 or ARMv6+LPAE, the non-shared device gets mapped to shared device in hardware. Looking through the arm32 code, it seems that
MT_DEVICE_NONSHAREDis used byarch/arm/mach-shmobile/setup-r8a7779.cand IIUC that's a v7 platform (R-Car H1, Cortex-A9). I think the above should be defined toL_PTE_MT_DEV_SHARED, unless I miss any place whereDEV_NONSHAREDis relevant on ARMv6 (adding Simon to confirm on shmbile)."
[1] https://lore.kernel.org/patchwork/patch/986574/
Cc Russell King <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Steve Capper <[email protected]>Cc Simon Horman <[email protected]>Cc Minchan Kim <[email protected]>Cc Suren Baghdasaryan <[email protected]>Signed-off-by Minchan Kim <[email protected]>Signed-off-by Miles Chen <[email protected]>--- arch/arm/include/asm/pgtable-2level.h | 21 +++++++++++++++++---- arch/arm/mm/proc-macros.S | 4 ++-- 2 files changed, 19 insertions(+), 6 deletions(-)arch/arm/include/asm/pgtable-2level.h +0
-0
arch/arm/include/asm/pgtable-2level.h line 167
Quoted 9 lines
arch/arm/include/asm/pgtable-2level.h +2
-2@@ -161,14 +161,27 @@#define L_PTE_MT_BUFFERABLE (_AT(pteval_t, 0x01) << 2)/* 0001 */ #define L_PTE_MT_WRITETHROUGH (_AT(pteval_t, 0x02) << 2)/* 0010 */ #define L_PTE_MT_WRITEBACK (_AT(pteval_t, 0x03) << 2)/* 0011 */ +#define L_PTE_MT_DEV_SHARED (_AT(pteval_t, 0x04) << 2)/* 0100 */ +#define L_PTE_MT_VECTORS (_AT(pteval_t, 0x05) << 2)/* 0101 */ #define L_PTE_MT_MINICACHE (_AT(pteval_t, 0x06) << 2)/* 0110 (sa1100, xscale) */ #define L_PTE_MT_WRITEALLOC (_AT(pteval_t, 0x07) << 2)/* 0111 */ -#define L_PTE_MT_DEV_SHARED (_AT(pteval_t, 0x04) << 2)/* 0100 */ -#define L_PTE_MT_DEV_NONSHARED (_AT(pteval_t, 0x0c) << 2)/* 1100 */
Sorry, no, this isn't going to work.
The lower two bits of this (bits 2 and 3) are explicitly designed to fit the C and B bits used in older architectures. Changing L_PTE_MT_VECTORS from having value '11' to '01' changes the functionality on older CPUs.
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
Re: [PATCH v2 2/4] arm: mm: reordering memory type table
Quoted 52 lines · Russell King - ARM Linux admin
Quoted 39 lines · Miles Chen
From: Minchan Kim <[email protected]>
To use bit 5 in page table as
L_PTE_SPECIAL, we need a room for that. It seems we don't need 4 bits for the memory type with ARMv6+. If it's true, let's reorder bits to make bit 5 free.We will use the bit for
L_PTE_SPECIALin next patch.A note from Catalin in [1]:
"Quoted 6 lines
Anyway, on ARMv7 or ARMv6+LPAE, the non-shared device gets mapped to shared device in hardware. Looking through the arm32 code, it seems that
MT_DEVICE_NONSHAREDis used byarch/arm/mach-shmobile/setup-r8a7779.cand IIUC that's a v7 platform (R-Car H1, Cortex-A9). I think the above should be defined toL_PTE_MT_DEV_SHARED, unless I miss any place whereDEV_NONSHAREDis relevant on ARMv6 (adding Simon to confirm on shmbile)."
[1] https://lore.kernel.org/patchwork/patch/986574/
Cc Russell King <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Steve Capper <[email protected]>Cc Simon Horman <[email protected]>Cc Minchan Kim <[email protected]>Cc Suren Baghdasaryan <[email protected]>Signed-off-by Minchan Kim <[email protected]>Signed-off-by Miles Chen <[email protected]>--- arch/arm/include/asm/pgtable-2level.h | 21 +++++++++++++++++---- arch/arm/mm/proc-macros.S | 4 ++-- 2 files changed, 19 insertions(+), 6 deletions(-)arch/arm/include/asm/pgtable-2level.h +0
-0arch/arm/include/asm/pgtable-2level.h line 167
Quoted 9 lines
arch/arm/include/asm/pgtable-2level.h +2
-2@@ -161,14 +161,27 @@#define L_PTE_MT_BUFFERABLE (_AT(pteval_t, 0x01) << 2)/* 0001 */ #define L_PTE_MT_WRITETHROUGH (_AT(pteval_t, 0x02) << 2)/* 0010 */ #define L_PTE_MT_WRITEBACK (_AT(pteval_t, 0x03) << 2)/* 0011 */ +#define L_PTE_MT_DEV_SHARED (_AT(pteval_t, 0x04) << 2)/* 0100 */ +#define L_PTE_MT_VECTORS (_AT(pteval_t, 0x05) << 2)/* 0101 */ #define L_PTE_MT_MINICACHE (_AT(pteval_t, 0x06) << 2)/* 0110 (sa1100, xscale) */ #define L_PTE_MT_WRITEALLOC (_AT(pteval_t, 0x07) << 2)/* 0111 */ -#define L_PTE_MT_DEV_SHARED (_AT(pteval_t, 0x04) << 2)/* 0100 */ -#define L_PTE_MT_DEV_NONSHARED (_AT(pteval_t, 0x0c) << 2)/* 1100 */ Sorry, no, this isn't going to work.
Quoted 4 lines
The lower two bits of this (bits 2 and 3) are explicitly designed to fit the C and B bits used in older architectures. Changing
L_PTE_MT_VECTORSfrom having value '11' to '01' changes the functionality on older CPUs.
thanks for the comment.
Is is possible to find other order to fit this? e.g.,
+#define L_PTE_MT_VECTORS (_AT(pteval_t, 0x07) << 2) /* 0111 */
#define L_PTE_MT_WRITEALLOC (_AT(pteval_t, 0x05) << 2) /* 0101 */
or only allow this types for the new CPUs?
Miles
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek
PATCH v2 3/4 arm: mm: introduce L_PTE_SPECIAL
In reply to Miles ChenFrom: Minchan Kim <[email protected]>
This patch introduces L_PTE_SPECIAL and pte functions for supporting get_user_pages_fast.
--- arch/arm/Kconfig | 4 ++-- arch/arm/include/asm/pgtable-2level.h | 1 + arch/arm/include/asm/pgtable-3level.h | 6 ------ arch/arm/include/asm/pgtable.h | 13 +++++++++++++ 4 files changed, 16 insertions(+), 8 deletions(-)
arch/arm/Kconfig +2 -2
@@ -13,7 +13,7 @@ config ARM select ARCH_HAS_KCOV select ARCH_HAS_MEMBARRIER_SYNC_CORE select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE - select ARCH_HAS_PTE_SPECIAL if ARM_LPAE + select ARCH_HAS_PTE_SPECIAL if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K) select ARCH_HAS_PHYS_TO_DMA select ARCH_HAS_SETUP_DMA_OPS select ARCH_HAS_SET_MEMORY
@@ -82,7 +82,7 @@ config ARM select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU select HAVE_EXIT_THREAD - select HAVE_FAST_GUP if ARM_LPAE + select HAVE_FAST_GUP if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K) select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL select HAVE_FUNCTION_GRAPH_TRACER if !THUMB2_KERNEL && !CC_IS_CLANG select HAVE_FUNCTION_TRACER if !XIP_KERNEL
arch/arm/include/asm/pgtable-2level.h +1 -0
@@ -117,6 +117,7 @@#define L_PTE_VALID (_AT(pteval_t, 1) << 0)/* Valid */ #define L_PTE_PRESENT (_AT(pteval_t, 1) << 0)#define L_PTE_YOUNG (_AT(pteval_t, 1) << 1) +#define L_PTE_SPECIAL (_AT(pteval_t, 1) << 5)#define L_PTE_DIRTY (_AT(pteval_t, 1) << 6)#define L_PTE_RDONLY (_AT(pteval_t, 1) << 7)#define L_PTE_USER (_AT(pteval_t, 1) << 8)
arch/arm/include/asm/pgtable-3level.h +0 -6
@@ -175,12 +175,6 @@ static inline pmd_t *pud_page_vaddr(pud_t pud)#define pmd_present (pmd) (pmd_isset((pmd), L_PMD_SECT_VALID))#define pmd_young (pmd) (pmd_isset((pmd), PMD_SECT_AF)) -#define pte_special (pte) (pte_isset((pte), L_PTE_SPECIAL)) -static inline pte_t pte_mkspecial (pte_t pte) -{ -pte_val (pte) |=L_PTE_SPECIAL ; -return pte; -}#define pmd_write (pmd) (pmd_isclear((pmd), L_PMD_SECT_RDONLY))#define pmd_dirty (pmd) (pmd_isset((pmd), L_PMD_SECT_DIRTY))
arch/arm/include/asm/pgtable.h +13 -0
@@ -195,6 +195,11 @@ static inline pte_t *pmd_page_vaddr(pmd_t pmd)#define pte_dirty (pte) (pte_isset((pte), L_PTE_DIRTY))#define pte_young (pte) (pte_isset((pte), L_PTE_YOUNG))#define pte_exec (pte) (pte_isclear((pte), L_PTE_XN)) +#ifdef CONFIG_ARCH_HAS_PTE_SPECIAL +#define pte_special (pte) (pte_isset((pte), L_PTE_SPECIAL)) +#else +#define pte_special (pte) (0) +#endif #define pte_valid_user (pte) \ (pte_valid(pte) && pte_isset((pte), L_PTE_USER) && pte_young(pte))
@@ -274,6 +279,14 @@ static inline pte_t pte_mknexec(pte_t pte)return set_pte_bit (pte,__pgprot (L_PTE_XN )); } +#ifdef CONFIG_ARCH_HAS_PTE_SPECIAL +static inline pte_t pte_mkspecial (pte_t pte) +{ +return set_pte_bit (pte,__pgprot (L_PTE_SPECIAL )); +} +#else +static inline pte_t pte_mkspecial (pte_t pte) {return pte; } +#endif static inline pte_t pte_modify (pte_t pte,pgprot_t newprot) { const pteval_t mask = L_PTE_XN | L_PTE_RDONLY | L_PTE_USER |
Signature · 1 line
2.18.0
Russell King - ARM Linux admin
maintainerRe: [PATCH v2 3/4] arm: mm: introduce L_PTE_SPECIAL
Quoted 47 lines · Miles Chen
From: Minchan Kim <[email protected]>
This patch introduces
L_PTE_SPECIALand pte functions for supportingget_user_pages_fast.Cc Russell King <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Steve Capper <[email protected]>Cc Minchan Kim <[email protected]>Cc Suren Baghdasaryan <[email protected]>Signed-off-by Minchan Kim <[email protected]>Signed-off-by Miles Chen <[email protected]>--- arch/arm/Kconfig | 4 ++-- arch/arm/include/asm/pgtable-2level.h | 1 + arch/arm/include/asm/pgtable-3level.h | 6 ------ arch/arm/include/asm/pgtable.h | 13 +++++++++++++ 4 files changed, 16 insertions(+), 8 deletions(-)arch/arm/Kconfig +2
-2@@ -13,7 +13,7 @@ config ARM select ARCH_HAS_KCOV select ARCH_HAS_MEMBARRIER_SYNC_CORE select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE - select ARCH_HAS_PTE_SPECIAL if ARM_LPAE + select ARCH_HAS_PTE_SPECIAL if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K) select ARCH_HAS_PHYS_TO_DMA select ARCH_HAS_SETUP_DMA_OPS select ARCH_HAS_SET_MEMORY@@ -82,7 +82,7 @@ config ARM select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU select HAVE_EXIT_THREAD - select HAVE_FAST_GUP if ARM_LPAE + select HAVE_FAST_GUP if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K) select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL select HAVE_FUNCTION_GRAPH_TRACER if !THUMB2_KERNEL && !CC_IS_CLANG select HAVE_FUNCTION_TRACER if !XIP_KERNELarch/arm/include/asm/pgtable-2level.h +0
-0
arch/arm/include/asm/pgtable-2level.h line 120
Quoted 4 lines
arch/arm/include/asm/pgtable-2level.h +1
-0@@ -117,6 +117,7 @@#define L_PTE_VALID (_AT(pteval_t, 1) << 0)/* Valid */ #define L_PTE_PRESENT (_AT(pteval_t, 1) << 0)#define L_PTE_YOUNG (_AT(pteval_t, 1) << 1) +#define L_PTE_SPECIAL (_AT(pteval_t, 1) << 5)
How does this work? Bits 2 through 5 are already in use for the memory type.
Repurposing this bit means that L_PTE_MT_DEV_NONSHARED, L_PTE_MT_DEV_WC, L_PTE_MT_DEV_CACHED and L_PTE_MT_VECTORS clash with it.
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
Re: [PATCH v2 3/4] arm: mm: introduce L_PTE_SPECIAL
Quoted 56 lines · Russell King - ARM Linux admin
Quoted 47 lines · Miles Chen
From: Minchan Kim <[email protected]>
This patch introduces
L_PTE_SPECIALand pte functions for supportingget_user_pages_fast.Cc Russell King <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Steve Capper <[email protected]>Cc Minchan Kim <[email protected]>Cc Suren Baghdasaryan <[email protected]>Signed-off-by Minchan Kim <[email protected]>Signed-off-by Miles Chen <[email protected]>--- arch/arm/Kconfig | 4 ++-- arch/arm/include/asm/pgtable-2level.h | 1 + arch/arm/include/asm/pgtable-3level.h | 6 ------ arch/arm/include/asm/pgtable.h | 13 +++++++++++++ 4 files changed, 16 insertions(+), 8 deletions(-)arch/arm/Kconfig +2
-2@@ -13,7 +13,7 @@ config ARM select ARCH_HAS_KCOV select ARCH_HAS_MEMBARRIER_SYNC_CORE select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE - select ARCH_HAS_PTE_SPECIAL if ARM_LPAE + select ARCH_HAS_PTE_SPECIAL if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K) select ARCH_HAS_PHYS_TO_DMA select ARCH_HAS_SETUP_DMA_OPS select ARCH_HAS_SET_MEMORY@@ -82,7 +82,7 @@ config ARM select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU select HAVE_EXIT_THREAD - select HAVE_FAST_GUP if ARM_LPAE + select HAVE_FAST_GUP if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K) select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL select HAVE_FUNCTION_GRAPH_TRACER if !THUMB2_KERNEL && !CC_IS_CLANG select HAVE_FUNCTION_TRACER if !XIP_KERNELarch/arm/include/asm/pgtable-2level.h +0
-0arch/arm/include/asm/pgtable-2level.h line 120
Quoted 4 lines
arch/arm/include/asm/pgtable-2level.h +1
-0@@ -117,6 +117,7 @@#define L_PTE_VALID (_AT(pteval_t, 1) << 0)/* Valid */ #define L_PTE_PRESENT (_AT(pteval_t, 1) << 0)#define L_PTE_YOUNG (_AT(pteval_t, 1) << 1) +#define L_PTE_SPECIAL (_AT(pteval_t, 1) << 5)How does this work? Bits 2 through 5 are already in use for the memory type.
Quoted 3 lines
Repurposing this bit means that
L_PTE_MT_DEV_NONSHARED,L_PTE_MT_DEV_WC,L_PTE_MT_DEV_CACHEDandL_PTE_MT_VECTORSclash with it.
Thanks for the comment.
The idea is to re-order the memory type table in [1] (patch v2/4) and use bit 5 for L_PTE_SPECIAL.
[1] https://lore.kernel.org/patchwork/patch/1323893/
Miles
Russell King - ARM Linux admin
maintainerRe: [PATCH v2 3/4] arm: mm: introduce L_PTE_SPECIAL
Quoted 57 lines · Miles Chen
Quoted 56 lines · Russell King - ARM Linux admin
Quoted 47 lines · Miles Chen
From: Minchan Kim <[email protected]>
This patch introduces
L_PTE_SPECIALand pte functions for supportingget_user_pages_fast.Cc Russell King <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Steve Capper <[email protected]>Cc Minchan Kim <[email protected]>Cc Suren Baghdasaryan <[email protected]>Signed-off-by Minchan Kim <[email protected]>Signed-off-by Miles Chen <[email protected]>--- arch/arm/Kconfig | 4 ++-- arch/arm/include/asm/pgtable-2level.h | 1 + arch/arm/include/asm/pgtable-3level.h | 6 ------ arch/arm/include/asm/pgtable.h | 13 +++++++++++++ 4 files changed, 16 insertions(+), 8 deletions(-)arch/arm/Kconfig +2
-2@@ -13,7 +13,7 @@ config ARM select ARCH_HAS_KCOV select ARCH_HAS_MEMBARRIER_SYNC_CORE select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE - select ARCH_HAS_PTE_SPECIAL if ARM_LPAE + select ARCH_HAS_PTE_SPECIAL if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K) select ARCH_HAS_PHYS_TO_DMA select ARCH_HAS_SETUP_DMA_OPS select ARCH_HAS_SET_MEMORY@@ -82,7 +82,7 @@ config ARM select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU select HAVE_EXIT_THREAD - select HAVE_FAST_GUP if ARM_LPAE + select HAVE_FAST_GUP if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K) select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL select HAVE_FUNCTION_GRAPH_TRACER if !THUMB2_KERNEL && !CC_IS_CLANG select HAVE_FUNCTION_TRACER if !XIP_KERNELarch/arm/include/asm/pgtable-2level.h +0
-0arch/arm/include/asm/pgtable-2level.h line 120
Quoted 4 lines
arch/arm/include/asm/pgtable-2level.h +1
-0@@ -117,6 +117,7 @@#define L_PTE_VALID (_AT(pteval_t, 1) << 0)/* Valid */ #define L_PTE_PRESENT (_AT(pteval_t, 1) << 0)#define L_PTE_YOUNG (_AT(pteval_t, 1) << 1) +#define L_PTE_SPECIAL (_AT(pteval_t, 1) << 5)How does this work? Bits 2 through 5 are already in use for the memory type.
Quoted 7 lines
Quoted 3 lines
Repurposing this bit means that
L_PTE_MT_DEV_NONSHARED,L_PTE_MT_DEV_WC,L_PTE_MT_DEV_CACHEDandL_PTE_MT_VECTORSclash with it.Thanks for the comment.
The idea is to re-order the memory type table in [1] (patch v2/4) and use bit 5 forL_PTE_SPECIAL.
Thanks, I know what you are trying to achieve. I don't think it's possible without breaking the kernel on some CPUs and configurations.
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
Re: [PATCH v2 3/4] arm: mm: introduce L_PTE_SPECIAL
Quoted 62 lines · Russell King - ARM Linux admin
Quoted 61 lines · Miles Chen
Quoted 59 lines · Russell King - ARM Linux admin
Quoted 47 lines · Miles Chen
From: Minchan Kim <[email protected]>
This patch introduces
L_PTE_SPECIALand pte functions for supportingget_user_pages_fast.Cc Russell King <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Steve Capper <[email protected]>Cc Minchan Kim <[email protected]>Cc Suren Baghdasaryan <[email protected]>Signed-off-by Minchan Kim <[email protected]>Signed-off-by Miles Chen <[email protected]>--- arch/arm/Kconfig | 4 ++-- arch/arm/include/asm/pgtable-2level.h | 1 + arch/arm/include/asm/pgtable-3level.h | 6 ------ arch/arm/include/asm/pgtable.h | 13 +++++++++++++ 4 files changed, 16 insertions(+), 8 deletions(-)arch/arm/Kconfig +2
-2@@ -13,7 +13,7 @@ config ARM select ARCH_HAS_KCOV select ARCH_HAS_MEMBARRIER_SYNC_CORE select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE - select ARCH_HAS_PTE_SPECIAL if ARM_LPAE + select ARCH_HAS_PTE_SPECIAL if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K) select ARCH_HAS_PHYS_TO_DMA select ARCH_HAS_SETUP_DMA_OPS select ARCH_HAS_SET_MEMORY@@ -82,7 +82,7 @@ config ARM select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU select HAVE_EXIT_THREAD - select HAVE_FAST_GUP if ARM_LPAE + select HAVE_FAST_GUP if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K) select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL select HAVE_FUNCTION_GRAPH_TRACER if !THUMB2_KERNEL && !CC_IS_CLANG select HAVE_FUNCTION_TRACER if !XIP_KERNELarch/arm/include/asm/pgtable-2level.h +0
-0arch/arm/include/asm/pgtable-2level.h line 120
Quoted 4 lines
arch/arm/include/asm/pgtable-2level.h +1
-0@@ -117,6 +117,7 @@#define L_PTE_VALID (_AT(pteval_t, 1) << 0)/* Valid */ #define L_PTE_PRESENT (_AT(pteval_t, 1) << 0)#define L_PTE_YOUNG (_AT(pteval_t, 1) << 1) +#define L_PTE_SPECIAL (_AT(pteval_t, 1) << 5)How does this work? Bits 2 through 5 are already in use for the memory type.
Repurposing this bit means that
L_PTE_MT_DEV_NONSHARED,L_PTE_MT_DEV_WC,L_PTE_MT_DEV_CACHEDandL_PTE_MT_VECTORSclash with it.
Quoted 7 lines
Quoted 3 lines
Thanks for the comment.
The idea is to re-order the memory type table in [1] (patch v2/4) and use bit 5 forL_PTE_SPECIAL.Thanks, I know what you are trying to achieve. I don't think it's possible without breaking the kernel on some CPUs and configurations.
Got it. Thanks for your review.
Miles
PATCH v2 4/4 arm: replace vector mem type with read-only type
In reply to Miles ChenSince kernel no longer writes to the vector, try to replace the vector mem type with read-only type and remove L_PTE_MT_VECTORS.
from Catalin in [1]:
"
Quoted 4 lines
I don't think this matters since the kernel no longer writes to the vectors page at run-time but it needs cleaning up a bit (and testing in case I missed something). IOW, do we still need a dedicated mapping type for the vectors or we can simply use the read-only user page attributes?
"
[1] https://lore.kernel.org/patchwork/patch/986574/
--- arch/arm/include/asm/pgtable-2level.h | 1 - arch/arm/mm/mmu.c | 7 +++---- arch/arm/mm/proc-macros.S | 2 +- 3 files changed, 4 insertions(+), 6 deletions(-)
arch/arm/include/asm/pgtable-2level.h +0 -1
@@ -163,7 +163,6 @@#define L_PTE_MT_WRITETHROUGH (_AT(pteval_t, 0x02) << 2)/* 0010 */ #define L_PTE_MT_WRITEBACK (_AT(pteval_t, 0x03) << 2)/* 0011 */ #define L_PTE_MT_DEV_SHARED (_AT(pteval_t, 0x04) << 2)/* 0100 */ -#define L_PTE_MT_VECTORS (_AT(pteval_t, 0x05) << 2)/* 0101 */ #define L_PTE_MT_MINICACHE (_AT(pteval_t, 0x06) << 2)/* 0110 (sa1100, xscale) */ #define L_PTE_MT_WRITEALLOC (_AT(pteval_t, 0x07) << 2)/* 0111 */ #if defined(CONFIG_CPU_V7 ) || defined(CONFIG_CPU_V6 ) || defined(CONFIG_CPU_V6K )
arch/arm/mm/mmu.c +3 -4
@@ -536,12 +536,11 @@ static void __init build_mem_type_table(void)#ifndef CONFIG_ARM_LPAE /* -* We don't use domains on ARMv6 (since this causes problems with -* v6/v7 kernels), so we must use a separate memory type for user -* r/o, kernel r/w to map the vectors page. +* We no longer write to the vector pages ar run-time, to +* use read-only user page attribute for vector. */ if (cpu_arch == CPU_ARCH_ARMv6) - vecs_pgprot |=L_PTE_MT_VECTORS ; + vecs_pgprot |=L_PTE_RDONLY ;/* * Check is it with support for the PXN bit
arch/arm/mm/proc-macros.S +1 -1
@@ -137,7 +137,7 @@ .long PTE_CACHEABLE @ L_PTE_MT_WRITETHROUGH .long PTE_CACHEABLE | PTE_BUFFERABLE @ L_PTE_MT_WRITEBACK .long PTE_BUFFERABLE @ L_PTE_MT_DEV_SHARED - .long PTE_CACHEABLE | PTE_BUFFERABLE | PTE_EXT_APX @ L_PTE_MT_VECTORS + .long 0x00 @ unused .long 0x00 @ L_PTE_MT_MINICACHE (not present) .long PTE_EXT_TEX(1) | PTE_CACHEABLE | PTE_BUFFERABLE @ L_PTE_MT_WRITEALLOC .long 0x00 @ unused
Signature · 1 line
2.18.0
Russell King - ARM Linux admin
maintainerRe: [PATCH v2 4/4] arm: replace vector mem type with read-only type
Quoted 10 lines · Miles Chen
Since kernel no longer writes to the vector, try to replace the vector mem type with read-only type and remove
L_PTE_MT_VECTORS.from Catalin in [1]:
"Quoted 4 lines
I don't think this matters since the kernel no longer writes to the vectors page at run-time but it needs cleaning up a bit (and testing in case I missed something). IOW, do we still need a dedicated mapping type for the vectors or we can simply use the read-only user page attributes?
"
Catalin is incorrect. If CONFIG_KUSER_HELPERS is enabled, then the vectors page is definitely written to - it's a user interface, so it's not going to change:
#ifdef CONFIG_KUSER_HELPERS
/*
* User space must never try to access this
* directly. Expect your app to break
* eventually if you do so. The user helper
* at 0xffff0fe0 must be used instead. (see
* entry-armv.S for details)
*/
*((unsigned int *)0xffff0ff0) = val;
#endif
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
Re: [PATCH v2 4/4] arm: replace vector mem type with read-only type
Quoted 16 lines · Russell King - ARM Linux admin
Quoted 10 lines · Miles Chen
Since kernel no longer writes to the vector, try to replace the vector mem type with read-only type and remove
L_PTE_MT_VECTORS.from Catalin in [1]:
"Quoted 4 lines
I don't think this matters since the kernel no longer writes to the vectors page at run-time but it needs cleaning up a bit (and testing in case I missed something). IOW, do we still need a dedicated mapping type for the vectors or we can simply use the read-only user page attributes?
"
Catalin is incorrect. If
CONFIG_KUSER_HELPERSis enabled, then the vectors page is definitely written to - it's a user interface, so it's not going to change:
Quoted 11 lines
#ifdef
CONFIG_KUSER_HELPERS
/*
* User space must never try to access this
* directly. Expect your app to break
* eventually if you do so. The user helper
* at0xffff0fe0must be used instead. (see
*entry-armv.Sfor details)
*/
*((unsigned int *)0xffff0ff0) = val;
#endif
Thanks for the comment.
We have to keep L_PTE_MT_VECTORS for KUSER_HELPERS.
Miles