Skip to contentSkip to conversation

arm: support get_user_pages_fast

PATCH In review Miles Chen 2 participants 13 messages Oct 23, 2020 – Nov 1, 2020

Subsystems (2)
lore
Share & tools
lore

Miles Chen

author
#0 lore raw

v2 cover arm: support get_user_pages_fast

get_user_pages_fast helps app's launching time because it could reduce mmap_sem lock contentions when app is launching.

Minchan posted a RFC patch and patch v1 [1] in 2018 and currently he has no bandwidth to rework this patch, so I rebase Minchan's patch (arm: support get_user_pages_fast) and modify the patch according to Catalin's comment and re-post this patch.

First, to use generic get_user_pages_fast code, use strict mm type in pgtable-2level-types.

2nd patch: reorder memory type table to use 5th bit of the page table

3rd patch: introduces L_PTE_SPECIAL for arm so that last patch can support get_user_pags_fast.

4th patch: remove L_PTE_MT_VECTORS and use read-only type.

[1] https://lore.kernel.org/patchwork/cover/986573/

Change since v1:
* modify comment
* remove L_PTE_MT_VECTORS and use read-only type

Change since RFC:
* Remove CPU_v7M - Catalin
* define DEV_NONSHARED to SHARED - Catalin
* https://lkml.kernel.org/r/[email protected]

Miles Chen (2):
arm: mm: use strict p[gum]d types
arm: replace vector mem type with read-only type

Minchan Kim (2):
arm: mm: reordering memory type table
arm: mm: introduce L_PTE_SPECIAL

 arch/arm/Kconfig                            |  4 ++--
 arch/arm/include/asm/fixmap.h               |  6 ++---
 arch/arm/include/asm/pgtable-2level-types.h | 26 ---------------------
 arch/arm/include/asm/pgtable-2level.h       | 22 +++++++++++++----
 arch/arm/include/asm/pgtable-3level.h       |  6 -----
 arch/arm/include/asm/pgtable.h              | 13 +++++++++++
 arch/arm/mm/mmu.c                           |  7 +++---
 arch/arm/mm/proc-macros.S                   |  2 +-
 8 files changed, 40 insertions(+), 46 deletions(-)
author
#1 lore raw

PATCH v2 1/4 arm: mm: use strict p[gum]d types

When use mm/gup.c, it is necessary to use strict p*d types. Otherwise, we will get build breaks such as:

mm/gup.c:2589:9: error: incompatible pointer types passing 'pmdval_t **' (aka 'unsigned int **') to parameter of type 'pgd_t *'
(aka 'pmdval_t (*)[2]') [-Werror,-Wincompatible-pointer-types] mm/gup.c:2616:9: error: array initializer must be an initializer list

Fix some types errors in fixmap.h after using strict types.

Cc Russell King <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
---
 arch/arm/include/asm/fixmap.h               |  6 ++---
 arch/arm/include/asm/pgtable-2level-types.h | 26 ---------------------
 arch/arm/include/asm/pgtable-2level.h       |  1 +
 3 files changed, 4 insertions(+), 29 deletions(-)
arch/arm/include/asm/fixmap.h +3 -3
index fc56fc3e1931..3ea12b2da708 100644
@@ -42,11 +42,11 @@ static const enum fixed_addresses __end_of_fixed_addresses =
 
 #define FIXMAP_PAGE_COMMON	(L_PTE_YOUNG | L_PTE_PRESENT | L_PTE_XN | L_PTE_DIRTY)
 
-#define FIXMAP_PAGE_NORMAL	(pgprot_kernel | L_PTE_XN)
-#define FIXMAP_PAGE_RO		(FIXMAP_PAGE_NORMAL | L_PTE_RDONLY)
+#define FIXMAP_PAGE_NORMAL	__pgprot(pgprot_val(pgprot_kernel) | L_PTE_XN)
+#define FIXMAP_PAGE_RO		__pgprot(pgprot_val(FIXMAP_PAGE_NORMAL) | L_PTE_RDONLY)
 
 /* Used by set_fixmap_(io|nocache), both meant for mapping a device */
-#define FIXMAP_PAGE_IO		(FIXMAP_PAGE_COMMON | L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
+#define FIXMAP_PAGE_IO		__pgprot(FIXMAP_PAGE_COMMON | L_PTE_MT_DEV_SHARED | L_PTE_SHARED)
 #define FIXMAP_PAGE_NOCACHE	FIXMAP_PAGE_IO
 
 #define __early_set_fixmap	__set_fixmap
arch/arm/include/asm/pgtable-2level-types.h +0 -26
index 650e793f4142..64fd5b8d9dc2 100644
@@ -12,12 +12,6 @@
 typedef u32 pteval_t;
 typedef u32 pmdval_t;
 
-#undef STRICT_MM_TYPECHECKS
-
-#ifdef STRICT_MM_TYPECHECKS
-/*
- * These are used to make use of C type-checking..
- */
 typedef struct { pteval_t pte; } pte_t;
 typedef struct { pmdval_t pmd; } pmd_t;
 typedef struct { pmdval_t pgd[2]; } pgd_t;
@@ -32,24 +26,4 @@ typedef struct { pteval_t pgprot; } pgprot_t;
 #define __pmd(x)        ((pmd_t) { (x) } )
 #define __pgprot(x)     ((pgprot_t) { (x) } )
 
-#else
-/*
- * .. while these make it easier on the compiler
- */
-typedef pteval_t pte_t;
-typedef pmdval_t pmd_t;
-typedef pmdval_t pgd_t[2];
-typedef pteval_t pgprot_t;
-
-#define pte_val(x)      (x)
-#define pmd_val(x)      (x)
-#define pgd_val(x)	((x)[0])
-#define pgprot_val(x)   (x)
-
-#define __pte(x)        (x)
-#define __pmd(x)        (x)
-#define __pgprot(x)     (x)
-
-#endif /* STRICT_MM_TYPECHECKS */
-
 #endif	/* _ASM_PGTABLE_2LEVEL_TYPES_H */
arch/arm/include/asm/pgtable-2level.h +1 -0
index 3502c2f746ca..27a8635abea0 100644
@@ -180,6 +180,7 @@
 #define pud_none(pud)		(0)
 #define pud_bad(pud)		(0)
 #define pud_present(pud)	(1)
+#define pud_page(pud)		pmd_page(__pmd(pud_val(pud)))
 #define pud_clear(pudp)		do { } while (0)
 #define set_pud(pud,pudp)	do { } while (0)
 
Signature · 1 line
2.18.0
author
#2 lore raw

PATCH v2 2/4 arm: mm: reordering memory type table

In reply to Miles Chen

From: Minchan Kim <[email protected]>

To use bit 5 in page table as L_PTE_SPECIAL, we need a room for that. It seems we don't need 4 bits for the memory type with ARMv6+. If it's true, let's reorder bits to make bit 5 free.

We will use the bit for L_PTE_SPECIAL in next patch.

A note from Catalin in [1]:
"

Quoted 6 lines

Anyway, on ARMv7 or ARMv6+LPAE, the non-shared device gets mapped to shared device in hardware. Looking through the arm32 code, it seems that MT_DEVICE_NONSHARED is used by arch/arm/mach-shmobile/setup-r8a7779.c and IIUC that's a v7 platform (R-Car H1, Cortex-A9). I think the above should be defined to L_PTE_MT_DEV_SHARED, unless I miss any place where DEV_NONSHARED is relevant on ARMv6 (adding Simon to confirm on shmbile).

"

[1] https://lore.kernel.org/patchwork/patch/986574/

Cc Russell King <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Steve Capper <[email protected]>
Cc Simon Horman <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
---
 arch/arm/include/asm/pgtable-2level.h | 21 +++++++++++++++++----
 arch/arm/mm/proc-macros.S             |  4 ++--
 2 files changed, 19 insertions(+), 6 deletions(-)
arch/arm/include/asm/pgtable-2level.h +17 -4
index 27a8635abea0..cdcd55cca37d 100644
@@ -161,14 +161,27 @@
 #define L_PTE_MT_BUFFERABLE	(_AT(pteval_t, 0x01) << 2)	/* 0001 */
 #define L_PTE_MT_WRITETHROUGH	(_AT(pteval_t, 0x02) << 2)	/* 0010 */
 #define L_PTE_MT_WRITEBACK	(_AT(pteval_t, 0x03) << 2)	/* 0011 */
+#define L_PTE_MT_DEV_SHARED	(_AT(pteval_t, 0x04) << 2)	/* 0100 */
+#define L_PTE_MT_VECTORS	(_AT(pteval_t, 0x05) << 2)	/* 0101 */
 #define L_PTE_MT_MINICACHE	(_AT(pteval_t, 0x06) << 2)	/* 0110 (sa1100, xscale) */
 #define L_PTE_MT_WRITEALLOC	(_AT(pteval_t, 0x07) << 2)	/* 0111 */
-#define L_PTE_MT_DEV_SHARED	(_AT(pteval_t, 0x04) << 2)	/* 0100 */
-#define L_PTE_MT_DEV_NONSHARED	(_AT(pteval_t, 0x0c) << 2)	/* 1100 */
+#if defined(CONFIG_CPU_V7) || defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_V6K)
+/*
+ * On ARMv7 or ARMv7+LPAE, the non-shared and shared device types get
+ * mapped to the same TEX remapping index. On classic ARMv7, the
+ * shareability is controlled by the PRRR[17:16] field, indexed by
+ * L_PTE_SHARED. On ARMv7+LPAE the device mapping is always shareable.
+ */
+#define L_PTE_MT_DEV_NONSHARED	L_PTE_MT_DEV_SHARED
+#define L_PTE_MT_DEV_WC		L_PTE_MT_BUFFERABLE
+#define L_PTE_MT_DEV_CACHED	L_PTE_MT_WRITEBACK
+#define L_PTE_MT_MASK		(_AT(pteval_t, 0x07) << 2)
+#else
 #define L_PTE_MT_DEV_WC		(_AT(pteval_t, 0x09) << 2)	/* 1001 */
 #define L_PTE_MT_DEV_CACHED	(_AT(pteval_t, 0x0b) << 2)	/* 1011 */
-#define L_PTE_MT_VECTORS	(_AT(pteval_t, 0x0f) << 2)	/* 1111 */
-#define L_PTE_MT_MASK		(_AT(pteval_t, 0x0f) << 2)
+#define L_PTE_MT_DEV_NONSHARED	(_AT(pteval_t, 0x0c) << 2)	/* 1100 */
+#define L_PTE_MT_MASK           (_AT(pteval_t, 0x0f) << 2)
+#endif
 
 #ifndef __ASSEMBLY__
 
arch/arm/mm/proc-macros.S +2 -2
index e2c743aa2eb2..dde1d6374250 100644
@@ -137,7 +137,7 @@
 	.long	PTE_CACHEABLE					@ L_PTE_MT_WRITETHROUGH
 	.long	PTE_CACHEABLE | PTE_BUFFERABLE			@ L_PTE_MT_WRITEBACK
 	.long	PTE_BUFFERABLE					@ L_PTE_MT_DEV_SHARED
-	.long	0x00						@ unused
+	.long	PTE_CACHEABLE | PTE_BUFFERABLE | PTE_EXT_APX	@ L_PTE_MT_VECTORS
 	.long	0x00						@ L_PTE_MT_MINICACHE (not present)
 	.long	PTE_EXT_TEX(1) | PTE_CACHEABLE | PTE_BUFFERABLE	@ L_PTE_MT_WRITEALLOC
 	.long	0x00						@ unused
@@ -147,7 +147,7 @@
 	.long	PTE_EXT_TEX(2)					@ L_PTE_MT_DEV_NONSHARED
 	.long	0x00						@ unused
 	.long	0x00						@ unused
-	.long	PTE_CACHEABLE | PTE_BUFFERABLE | PTE_EXT_APX	@ L_PTE_MT_VECTORS
+	.long	0x00						@ unused
 	.endm
 
 	.macro	armv6_set_pte_ext pfx
Signature · 1 line
2.18.0
#3 lore raw

Re: [PATCH v2 2/4] arm: mm: reordering memory type table

Quoted 39 lines · Miles Chen

From: Minchan Kim <[email protected]>

To use bit 5 in page table as L_PTE_SPECIAL, we need a room for that. It seems we don't need 4 bits for the memory type with ARMv6+. If it's true, let's reorder bits to make bit 5 free.

We will use the bit for L_PTE_SPECIAL in next patch.

A note from Catalin in [1]:
"

Quoted 6 lines

Anyway, on ARMv7 or ARMv6+LPAE, the non-shared device gets mapped to shared device in hardware. Looking through the arm32 code, it seems that MT_DEVICE_NONSHARED is used by arch/arm/mach-shmobile/setup-r8a7779.c and IIUC that's a v7 platform (R-Car H1, Cortex-A9). I think the above should be defined to L_PTE_MT_DEV_SHARED, unless I miss any place where DEV_NONSHARED is relevant on ARMv6 (adding Simon to confirm on shmbile).

"

[1] https://lore.kernel.org/patchwork/patch/986574/

Cc Russell King <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Steve Capper <[email protected]>
Cc Simon Horman <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
---
 arch/arm/include/asm/pgtable-2level.h | 21 +++++++++++++++++----
 arch/arm/mm/proc-macros.S             |  4 ++--
 2 files changed, 19 insertions(+), 6 deletions(-)
arch/arm/include/asm/pgtable-2level.h +0 -0
index 27a8635abea0..cdcd55cca37d 100644

arch/arm/include/asm/pgtable-2level.h line 167

Quoted 9 lines
arch/arm/include/asm/pgtable-2level.h +2 -2
@@ -161,14 +161,27 @@
 #define L_PTE_MT_BUFFERABLE	(_AT(pteval_t, 0x01) << 2)	/* 0001 */
 #define L_PTE_MT_WRITETHROUGH	(_AT(pteval_t, 0x02) << 2)	/* 0010 */
 #define L_PTE_MT_WRITEBACK	(_AT(pteval_t, 0x03) << 2)	/* 0011 */
+#define L_PTE_MT_DEV_SHARED	(_AT(pteval_t, 0x04) << 2)	/* 0100 */
+#define L_PTE_MT_VECTORS	(_AT(pteval_t, 0x05) << 2)	/* 0101 */
 #define L_PTE_MT_MINICACHE	(_AT(pteval_t, 0x06) << 2)	/* 0110 (sa1100, xscale) */
 #define L_PTE_MT_WRITEALLOC	(_AT(pteval_t, 0x07) << 2)	/* 0111 */
-#define L_PTE_MT_DEV_SHARED	(_AT(pteval_t, 0x04) << 2)	/* 0100 */
-#define L_PTE_MT_DEV_NONSHARED	(_AT(pteval_t, 0x0c) << 2)	/* 1100 */

Sorry, no, this isn't going to work.

The lower two bits of this (bits 2 and 3) are explicitly designed to fit the C and B bits used in older architectures. Changing L_PTE_MT_VECTORS from having value '11' to '01' changes the functionality on older CPUs.

Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
author
#4 lore raw

Re: [PATCH v2 2/4] arm: mm: reordering memory type table

Quoted 52 lines · Russell King - ARM Linux admin
Quoted 39 lines · Miles Chen

From: Minchan Kim <[email protected]>

To use bit 5 in page table as L_PTE_SPECIAL, we need a room for that. It seems we don't need 4 bits for the memory type with ARMv6+. If it's true, let's reorder bits to make bit 5 free.

We will use the bit for L_PTE_SPECIAL in next patch.

A note from Catalin in [1]:
"

Quoted 6 lines

Anyway, on ARMv7 or ARMv6+LPAE, the non-shared device gets mapped to shared device in hardware. Looking through the arm32 code, it seems that MT_DEVICE_NONSHARED is used by arch/arm/mach-shmobile/setup-r8a7779.c and IIUC that's a v7 platform (R-Car H1, Cortex-A9). I think the above should be defined to L_PTE_MT_DEV_SHARED, unless I miss any place where DEV_NONSHARED is relevant on ARMv6 (adding Simon to confirm on shmbile).

"

[1] https://lore.kernel.org/patchwork/patch/986574/

Cc Russell King <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Steve Capper <[email protected]>
Cc Simon Horman <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
---
 arch/arm/include/asm/pgtable-2level.h | 21 +++++++++++++++++----
 arch/arm/mm/proc-macros.S             |  4 ++--
 2 files changed, 19 insertions(+), 6 deletions(-)
arch/arm/include/asm/pgtable-2level.h +0 -0
index 27a8635abea0..cdcd55cca37d 100644

arch/arm/include/asm/pgtable-2level.h line 167

Quoted 9 lines
arch/arm/include/asm/pgtable-2level.h +2 -2
@@ -161,14 +161,27 @@
 #define L_PTE_MT_BUFFERABLE	(_AT(pteval_t, 0x01) << 2)	/* 0001 */
 #define L_PTE_MT_WRITETHROUGH	(_AT(pteval_t, 0x02) << 2)	/* 0010 */
 #define L_PTE_MT_WRITEBACK	(_AT(pteval_t, 0x03) << 2)	/* 0011 */
+#define L_PTE_MT_DEV_SHARED	(_AT(pteval_t, 0x04) << 2)	/* 0100 */
+#define L_PTE_MT_VECTORS	(_AT(pteval_t, 0x05) << 2)	/* 0101 */
 #define L_PTE_MT_MINICACHE	(_AT(pteval_t, 0x06) << 2)	/* 0110 (sa1100, xscale) */
 #define L_PTE_MT_WRITEALLOC	(_AT(pteval_t, 0x07) << 2)	/* 0111 */
-#define L_PTE_MT_DEV_SHARED	(_AT(pteval_t, 0x04) << 2)	/* 0100 */
-#define L_PTE_MT_DEV_NONSHARED	(_AT(pteval_t, 0x0c) << 2)	/* 1100 */

Sorry, no, this isn't going to work.

Quoted 4 lines

The lower two bits of this (bits 2 and 3) are explicitly designed to fit the C and B bits used in older architectures. Changing L_PTE_MT_VECTORS from having value '11' to '01' changes the functionality on older CPUs.

thanks for the comment.

Is is possible to find other order to fit this? e.g.,

+#define L_PTE_MT_VECTORS (_AT(pteval_t, 0x07) << 2) /* 0111 */

#define L_PTE_MT_WRITEALLOC (_AT(pteval_t, 0x05) << 2) /* 0101 */

or only allow this types for the new CPUs?

Miles
_______________________________________________
Linux-mediatek mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-mediatek

author
#5 lore raw

PATCH v2 3/4 arm: mm: introduce L_PTE_SPECIAL

In reply to Miles Chen

From: Minchan Kim <[email protected]>

This patch introduces L_PTE_SPECIAL and pte functions for supporting get_user_pages_fast.

Cc Russell King <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Steve Capper <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
---
 arch/arm/Kconfig                      |  4 ++--
 arch/arm/include/asm/pgtable-2level.h |  1 +
 arch/arm/include/asm/pgtable-3level.h |  6 ------
 arch/arm/include/asm/pgtable.h        | 13 +++++++++++++
 4 files changed, 16 insertions(+), 8 deletions(-)
arch/arm/Kconfig +2 -2
index c18fa9d382b7..1f75864b7c7a 100644
@@ -13,7 +13,7 @@ config ARM
 	select ARCH_HAS_KCOV
 	select ARCH_HAS_MEMBARRIER_SYNC_CORE
 	select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
-	select ARCH_HAS_PTE_SPECIAL if ARM_LPAE
+	select ARCH_HAS_PTE_SPECIAL if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K)
 	select ARCH_HAS_PHYS_TO_DMA
 	select ARCH_HAS_SETUP_DMA_OPS
 	select ARCH_HAS_SET_MEMORY
@@ -82,7 +82,7 @@ config ARM
 	select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
 	select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU
 	select HAVE_EXIT_THREAD
-	select HAVE_FAST_GUP if ARM_LPAE
+	select HAVE_FAST_GUP if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K)
 	select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
 	select HAVE_FUNCTION_GRAPH_TRACER if !THUMB2_KERNEL && !CC_IS_CLANG
 	select HAVE_FUNCTION_TRACER if !XIP_KERNEL
arch/arm/include/asm/pgtable-2level.h +1 -0
index cdcd55cca37d..385e7a32394e 100644
@@ -117,6 +117,7 @@
 #define L_PTE_VALID		(_AT(pteval_t, 1) << 0)		/* Valid */
 #define L_PTE_PRESENT		(_AT(pteval_t, 1) << 0)
 #define L_PTE_YOUNG		(_AT(pteval_t, 1) << 1)
+#define L_PTE_SPECIAL		(_AT(pteval_t, 1) << 5)
 #define L_PTE_DIRTY		(_AT(pteval_t, 1) << 6)
 #define L_PTE_RDONLY		(_AT(pteval_t, 1) << 7)
 #define L_PTE_USER		(_AT(pteval_t, 1) << 8)
arch/arm/include/asm/pgtable-3level.h +0 -6
index fbb6693c3352..46fcc6725d3e 100644
@@ -175,12 +175,6 @@ static inline pmd_t *pud_page_vaddr(pud_t pud)
 
 #define pmd_present(pmd)	(pmd_isset((pmd), L_PMD_SECT_VALID))
 #define pmd_young(pmd)		(pmd_isset((pmd), PMD_SECT_AF))
-#define pte_special(pte)	(pte_isset((pte), L_PTE_SPECIAL))
-static inline pte_t pte_mkspecial(pte_t pte)
-{
-	pte_val(pte) |= L_PTE_SPECIAL;
-	return pte;
-}
 
 #define pmd_write(pmd)		(pmd_isclear((pmd), L_PMD_SECT_RDONLY))
 #define pmd_dirty(pmd)		(pmd_isset((pmd), L_PMD_SECT_DIRTY))
arch/arm/include/asm/pgtable.h +13 -0
index c02f24400369..4092154ca779 100644
@@ -195,6 +195,11 @@ static inline pte_t *pmd_page_vaddr(pmd_t pmd)
 #define pte_dirty(pte)		(pte_isset((pte), L_PTE_DIRTY))
 #define pte_young(pte)		(pte_isset((pte), L_PTE_YOUNG))
 #define pte_exec(pte)		(pte_isclear((pte), L_PTE_XN))
+#ifdef CONFIG_ARCH_HAS_PTE_SPECIAL
+#define pte_special(pte)	(pte_isset((pte), L_PTE_SPECIAL))
+#else
+#define pte_special(pte)	(0)
+#endif
 
 #define pte_valid_user(pte)	\
 	(pte_valid(pte) && pte_isset((pte), L_PTE_USER) && pte_young(pte))
@@ -274,6 +279,14 @@ static inline pte_t pte_mknexec(pte_t pte)
 	return set_pte_bit(pte, __pgprot(L_PTE_XN));
 }
 
+#ifdef CONFIG_ARCH_HAS_PTE_SPECIAL
+static inline pte_t pte_mkspecial(pte_t pte)
+{
+	return set_pte_bit(pte, __pgprot(L_PTE_SPECIAL));
+}
+#else
+static inline pte_t pte_mkspecial(pte_t pte) { return pte; }
+#endif
 static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
 {
 	const pteval_t mask = L_PTE_XN | L_PTE_RDONLY | L_PTE_USER |
Signature · 1 line
2.18.0
#6 lore raw

Re: [PATCH v2 3/4] arm: mm: introduce L_PTE_SPECIAL

Quoted 47 lines · Miles Chen

From: Minchan Kim <[email protected]>

This patch introduces L_PTE_SPECIAL and pte functions for supporting get_user_pages_fast.

Cc Russell King <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Steve Capper <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
---
 arch/arm/Kconfig                      |  4 ++--
 arch/arm/include/asm/pgtable-2level.h |  1 +
 arch/arm/include/asm/pgtable-3level.h |  6 ------
 arch/arm/include/asm/pgtable.h        | 13 +++++++++++++
 4 files changed, 16 insertions(+), 8 deletions(-)
arch/arm/Kconfig +2 -2
index c18fa9d382b7..1f75864b7c7a 100644
@@ -13,7 +13,7 @@ config ARM
 	select ARCH_HAS_KCOV
 	select ARCH_HAS_MEMBARRIER_SYNC_CORE
 	select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
-	select ARCH_HAS_PTE_SPECIAL if ARM_LPAE
+	select ARCH_HAS_PTE_SPECIAL if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K)
 	select ARCH_HAS_PHYS_TO_DMA
 	select ARCH_HAS_SETUP_DMA_OPS
 	select ARCH_HAS_SET_MEMORY
@@ -82,7 +82,7 @@ config ARM
 	select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
 	select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU
 	select HAVE_EXIT_THREAD
-	select HAVE_FAST_GUP if ARM_LPAE
+	select HAVE_FAST_GUP if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K)
 	select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
 	select HAVE_FUNCTION_GRAPH_TRACER if !THUMB2_KERNEL && !CC_IS_CLANG
 	select HAVE_FUNCTION_TRACER if !XIP_KERNEL
arch/arm/include/asm/pgtable-2level.h +0 -0
index cdcd55cca37d..385e7a32394e 100644

arch/arm/include/asm/pgtable-2level.h line 120

Quoted 4 lines
arch/arm/include/asm/pgtable-2level.h +1 -0
@@ -117,6 +117,7 @@
 #define L_PTE_VALID		(_AT(pteval_t, 1) << 0)		/* Valid */
 #define L_PTE_PRESENT		(_AT(pteval_t, 1) << 0)
 #define L_PTE_YOUNG		(_AT(pteval_t, 1) << 1)
+#define L_PTE_SPECIAL		(_AT(pteval_t, 1) << 5)

How does this work? Bits 2 through 5 are already in use for the memory type.

Repurposing this bit means that L_PTE_MT_DEV_NONSHARED, L_PTE_MT_DEV_WC, L_PTE_MT_DEV_CACHED and L_PTE_MT_VECTORS clash with it.

Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
author
#7 lore raw

Re: [PATCH v2 3/4] arm: mm: introduce L_PTE_SPECIAL

Quoted 56 lines · Russell King - ARM Linux admin
Quoted 47 lines · Miles Chen

From: Minchan Kim <[email protected]>

This patch introduces L_PTE_SPECIAL and pte functions for supporting get_user_pages_fast.

Cc Russell King <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Steve Capper <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
---
 arch/arm/Kconfig                      |  4 ++--
 arch/arm/include/asm/pgtable-2level.h |  1 +
 arch/arm/include/asm/pgtable-3level.h |  6 ------
 arch/arm/include/asm/pgtable.h        | 13 +++++++++++++
 4 files changed, 16 insertions(+), 8 deletions(-)
arch/arm/Kconfig +2 -2
index c18fa9d382b7..1f75864b7c7a 100644
@@ -13,7 +13,7 @@ config ARM
 	select ARCH_HAS_KCOV
 	select ARCH_HAS_MEMBARRIER_SYNC_CORE
 	select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
-	select ARCH_HAS_PTE_SPECIAL if ARM_LPAE
+	select ARCH_HAS_PTE_SPECIAL if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K)
 	select ARCH_HAS_PHYS_TO_DMA
 	select ARCH_HAS_SETUP_DMA_OPS
 	select ARCH_HAS_SET_MEMORY
@@ -82,7 +82,7 @@ config ARM
 	select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
 	select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU
 	select HAVE_EXIT_THREAD
-	select HAVE_FAST_GUP if ARM_LPAE
+	select HAVE_FAST_GUP if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K)
 	select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
 	select HAVE_FUNCTION_GRAPH_TRACER if !THUMB2_KERNEL && !CC_IS_CLANG
 	select HAVE_FUNCTION_TRACER if !XIP_KERNEL
arch/arm/include/asm/pgtable-2level.h +0 -0
index cdcd55cca37d..385e7a32394e 100644

arch/arm/include/asm/pgtable-2level.h line 120

Quoted 4 lines
arch/arm/include/asm/pgtable-2level.h +1 -0
@@ -117,6 +117,7 @@
 #define L_PTE_VALID		(_AT(pteval_t, 1) << 0)		/* Valid */
 #define L_PTE_PRESENT		(_AT(pteval_t, 1) << 0)
 #define L_PTE_YOUNG		(_AT(pteval_t, 1) << 1)
+#define L_PTE_SPECIAL		(_AT(pteval_t, 1) << 5)

How does this work? Bits 2 through 5 are already in use for the memory type.

Quoted 3 lines

Repurposing this bit means that L_PTE_MT_DEV_NONSHARED, L_PTE_MT_DEV_WC, L_PTE_MT_DEV_CACHED and L_PTE_MT_VECTORS clash with it.

Thanks for the comment.
The idea is to re-order the memory type table in [1] (patch v2/4) and use bit 5 for L_PTE_SPECIAL.

[1] https://lore.kernel.org/patchwork/patch/1323893/

Miles

#8 lore raw

Re: [PATCH v2 3/4] arm: mm: introduce L_PTE_SPECIAL

Quoted 57 lines · Miles Chen
Quoted 56 lines · Russell King - ARM Linux admin
Quoted 47 lines · Miles Chen

From: Minchan Kim <[email protected]>

This patch introduces L_PTE_SPECIAL and pte functions for supporting get_user_pages_fast.

Cc Russell King <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Steve Capper <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
---
 arch/arm/Kconfig                      |  4 ++--
 arch/arm/include/asm/pgtable-2level.h |  1 +
 arch/arm/include/asm/pgtable-3level.h |  6 ------
 arch/arm/include/asm/pgtable.h        | 13 +++++++++++++
 4 files changed, 16 insertions(+), 8 deletions(-)
arch/arm/Kconfig +2 -2
index c18fa9d382b7..1f75864b7c7a 100644
@@ -13,7 +13,7 @@ config ARM
 	select ARCH_HAS_KCOV
 	select ARCH_HAS_MEMBARRIER_SYNC_CORE
 	select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
-	select ARCH_HAS_PTE_SPECIAL if ARM_LPAE
+	select ARCH_HAS_PTE_SPECIAL if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K)
 	select ARCH_HAS_PHYS_TO_DMA
 	select ARCH_HAS_SETUP_DMA_OPS
 	select ARCH_HAS_SET_MEMORY
@@ -82,7 +82,7 @@ config ARM
 	select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
 	select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU
 	select HAVE_EXIT_THREAD
-	select HAVE_FAST_GUP if ARM_LPAE
+	select HAVE_FAST_GUP if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K)
 	select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
 	select HAVE_FUNCTION_GRAPH_TRACER if !THUMB2_KERNEL && !CC_IS_CLANG
 	select HAVE_FUNCTION_TRACER if !XIP_KERNEL
arch/arm/include/asm/pgtable-2level.h +0 -0
index cdcd55cca37d..385e7a32394e 100644

arch/arm/include/asm/pgtable-2level.h line 120

Quoted 4 lines
arch/arm/include/asm/pgtable-2level.h +1 -0
@@ -117,6 +117,7 @@
 #define L_PTE_VALID		(_AT(pteval_t, 1) << 0)		/* Valid */
 #define L_PTE_PRESENT		(_AT(pteval_t, 1) << 0)
 #define L_PTE_YOUNG		(_AT(pteval_t, 1) << 1)
+#define L_PTE_SPECIAL		(_AT(pteval_t, 1) << 5)

How does this work? Bits 2 through 5 are already in use for the memory type.

Quoted 7 lines
Quoted 3 lines

Repurposing this bit means that L_PTE_MT_DEV_NONSHARED, L_PTE_MT_DEV_WC, L_PTE_MT_DEV_CACHED and L_PTE_MT_VECTORS clash with it.

Thanks for the comment.
The idea is to re-order the memory type table in [1] (patch v2/4) and use bit 5 for L_PTE_SPECIAL.

Thanks, I know what you are trying to achieve. I don't think it's possible without breaking the kernel on some CPUs and configurations.

Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
author
#9 lore raw

Re: [PATCH v2 3/4] arm: mm: introduce L_PTE_SPECIAL

Quoted 62 lines · Russell King - ARM Linux admin
Quoted 61 lines · Miles Chen
Quoted 59 lines · Russell King - ARM Linux admin
Quoted 47 lines · Miles Chen

From: Minchan Kim <[email protected]>

This patch introduces L_PTE_SPECIAL and pte functions for supporting get_user_pages_fast.

Cc Russell King <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Will Deacon <[email protected]>
Cc Steve Capper <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
---
 arch/arm/Kconfig                      |  4 ++--
 arch/arm/include/asm/pgtable-2level.h |  1 +
 arch/arm/include/asm/pgtable-3level.h |  6 ------
 arch/arm/include/asm/pgtable.h        | 13 +++++++++++++
 4 files changed, 16 insertions(+), 8 deletions(-)
arch/arm/Kconfig +2 -2
index c18fa9d382b7..1f75864b7c7a 100644
@@ -13,7 +13,7 @@ config ARM
 	select ARCH_HAS_KCOV
 	select ARCH_HAS_MEMBARRIER_SYNC_CORE
 	select ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
-	select ARCH_HAS_PTE_SPECIAL if ARM_LPAE
+	select ARCH_HAS_PTE_SPECIAL if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K)
 	select ARCH_HAS_PHYS_TO_DMA
 	select ARCH_HAS_SETUP_DMA_OPS
 	select ARCH_HAS_SET_MEMORY
@@ -82,7 +82,7 @@ config ARM
 	select HAVE_DYNAMIC_FTRACE_WITH_REGS if HAVE_DYNAMIC_FTRACE
 	select HAVE_EFFICIENT_UNALIGNED_ACCESS if (CPU_V6 || CPU_V6K || CPU_V7) && MMU
 	select HAVE_EXIT_THREAD
-	select HAVE_FAST_GUP if ARM_LPAE
+	select HAVE_FAST_GUP if (ARM_LPAE || CPU_V7 || CPU_V6 || CPUV6K)
 	select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL
 	select HAVE_FUNCTION_GRAPH_TRACER if !THUMB2_KERNEL && !CC_IS_CLANG
 	select HAVE_FUNCTION_TRACER if !XIP_KERNEL
arch/arm/include/asm/pgtable-2level.h +0 -0
index cdcd55cca37d..385e7a32394e 100644

arch/arm/include/asm/pgtable-2level.h line 120

Quoted 4 lines
arch/arm/include/asm/pgtable-2level.h +1 -0
@@ -117,6 +117,7 @@
 #define L_PTE_VALID		(_AT(pteval_t, 1) << 0)		/* Valid */
 #define L_PTE_PRESENT		(_AT(pteval_t, 1) << 0)
 #define L_PTE_YOUNG		(_AT(pteval_t, 1) << 1)
+#define L_PTE_SPECIAL		(_AT(pteval_t, 1) << 5)

How does this work? Bits 2 through 5 are already in use for the memory type.

Repurposing this bit means that L_PTE_MT_DEV_NONSHARED, L_PTE_MT_DEV_WC, L_PTE_MT_DEV_CACHED and L_PTE_MT_VECTORS clash with it.

Quoted 7 lines
Quoted 3 lines

Thanks for the comment.
The idea is to re-order the memory type table in [1] (patch v2/4) and use bit 5 for L_PTE_SPECIAL.

Thanks, I know what you are trying to achieve. I don't think it's possible without breaking the kernel on some CPUs and configurations.

Got it. Thanks for your review.

Miles

author
#10 lore raw

PATCH v2 4/4 arm: replace vector mem type with read-only type

In reply to Miles Chen

Since kernel no longer writes to the vector, try to replace the vector mem type with read-only type and remove L_PTE_MT_VECTORS.

from Catalin in [1]:
"

Quoted 4 lines

I don't think this matters since the kernel no longer writes to the vectors page at run-time but it needs cleaning up a bit (and testing in case I missed something). IOW, do we still need a dedicated mapping type for the vectors or we can simply use the read-only user page attributes?

"

[1] https://lore.kernel.org/patchwork/patch/986574/

Cc Russell King <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc Minchan Kim <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
---
 arch/arm/include/asm/pgtable-2level.h | 1 -
 arch/arm/mm/mmu.c                     | 7 +++----
 arch/arm/mm/proc-macros.S             | 2 +-
 3 files changed, 4 insertions(+), 6 deletions(-)
arch/arm/include/asm/pgtable-2level.h +0 -1
index 385e7a32394e..438359d3675f 100644
@@ -163,7 +163,6 @@
 #define L_PTE_MT_WRITETHROUGH	(_AT(pteval_t, 0x02) << 2)	/* 0010 */
 #define L_PTE_MT_WRITEBACK	(_AT(pteval_t, 0x03) << 2)	/* 0011 */
 #define L_PTE_MT_DEV_SHARED	(_AT(pteval_t, 0x04) << 2)	/* 0100 */
-#define L_PTE_MT_VECTORS	(_AT(pteval_t, 0x05) << 2)	/* 0101 */
 #define L_PTE_MT_MINICACHE	(_AT(pteval_t, 0x06) << 2)	/* 0110 (sa1100, xscale) */
 #define L_PTE_MT_WRITEALLOC	(_AT(pteval_t, 0x07) << 2)	/* 0111 */
 #if defined(CONFIG_CPU_V7) || defined(CONFIG_CPU_V6) || defined(CONFIG_CPU_V6K)
arch/arm/mm/mmu.c +3 -4
index ab69250a86bc..0b6b377e2cce 100644
@@ -536,12 +536,11 @@ static void __init build_mem_type_table(void)
 
 #ifndef CONFIG_ARM_LPAE
 	/*
-	 * We don't use domains on ARMv6 (since this causes problems with
-	 * v6/v7 kernels), so we must use a separate memory type for user
-	 * r/o, kernel r/w to map the vectors page.
+	 * We no longer write to the vector pages ar run-time, to
+	 * use read-only user page attribute for vector.
 	 */
 	if (cpu_arch == CPU_ARCH_ARMv6)
-		vecs_pgprot |= L_PTE_MT_VECTORS;
+		vecs_pgprot |= L_PTE_RDONLY;
 
 	/*
 	 * Check is it with support for the PXN bit
arch/arm/mm/proc-macros.S +1 -1
index dde1d6374250..f3e6551b4a7e 100644
@@ -137,7 +137,7 @@
 	.long	PTE_CACHEABLE					@ L_PTE_MT_WRITETHROUGH
 	.long	PTE_CACHEABLE | PTE_BUFFERABLE			@ L_PTE_MT_WRITEBACK
 	.long	PTE_BUFFERABLE					@ L_PTE_MT_DEV_SHARED
-	.long	PTE_CACHEABLE | PTE_BUFFERABLE | PTE_EXT_APX	@ L_PTE_MT_VECTORS
+	.long	0x00						@ unused
 	.long	0x00						@ L_PTE_MT_MINICACHE (not present)
 	.long	PTE_EXT_TEX(1) | PTE_CACHEABLE | PTE_BUFFERABLE	@ L_PTE_MT_WRITEALLOC
 	.long	0x00						@ unused
Signature · 1 line
2.18.0
#11 lore raw

Re: [PATCH v2 4/4] arm: replace vector mem type with read-only type

Quoted 10 lines · Miles Chen

Since kernel no longer writes to the vector, try to replace the vector mem type with read-only type and remove L_PTE_MT_VECTORS.

from Catalin in [1]:
"

Quoted 4 lines

I don't think this matters since the kernel no longer writes to the vectors page at run-time but it needs cleaning up a bit (and testing in case I missed something). IOW, do we still need a dedicated mapping type for the vectors or we can simply use the read-only user page attributes?

"

Catalin is incorrect. If CONFIG_KUSER_HELPERS is enabled, then the vectors page is definitely written to - it's a user interface, so it's not going to change:

#ifdef CONFIG_KUSER_HELPERS
/*
* User space must never try to access this
* directly. Expect your app to break
* eventually if you do so. The user helper
* at 0xffff0fe0 must be used instead. (see
* entry-armv.S for details)
*/
*((unsigned int *)0xffff0ff0) = val;
#endif

Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
author
#12 lore raw

Re: [PATCH v2 4/4] arm: replace vector mem type with read-only type

Quoted 16 lines · Russell King - ARM Linux admin
Quoted 10 lines · Miles Chen

Since kernel no longer writes to the vector, try to replace the vector mem type with read-only type and remove L_PTE_MT_VECTORS.

from Catalin in [1]:
"

Quoted 4 lines

I don't think this matters since the kernel no longer writes to the vectors page at run-time but it needs cleaning up a bit (and testing in case I missed something). IOW, do we still need a dedicated mapping type for the vectors or we can simply use the read-only user page attributes?

"

Catalin is incorrect. If CONFIG_KUSER_HELPERS is enabled, then the vectors page is definitely written to - it's a user interface, so it's not going to change:

Quoted 11 lines

#ifdef CONFIG_KUSER_HELPERS
/*
* User space must never try to access this
* directly. Expect your app to break
* eventually if you do so. The user helper
* at 0xffff0fe0 must be used instead. (see
* entry-armv.S for details)
*/
*((unsigned int *)0xffff0ff0) = val;
#endif

Thanks for the comment.
We have to keep L_PTE_MT_VECTORS for KUSER_HELPERS.

Miles