enable PT_RECLAIM on all 64-bit architectures
Subsystems (10)
v2 cover enable PT_RECLAIM on all 64-bit architectures
From: Qi Zheng <[email protected]>
Changelog in v2:
- fix compilation errors (reported by Magnus Lindholm and kernel test robot)
- adjust some code style (suggested by Huacai Chen)
- make PT_RECLAIM user-unselectable (suggested by David Hildenbrand)
- rebase onto the next-20251119
Hi all,
This series aims to enable PT_RECLAIM on all 64-bit architectures.
On a 64-bit system, madvise(MADV_DONTNEED) may cause a large number of empty PTE page table pages (such as 100GB+). To resolve this problem, we need to enable PT_RECLAIM, which depends on MMU_GATHER_RCU_TABLE_FREE.
Therefore, this series first enables MMU_GATHER_RCU_TABLE_FREE on all 64-bit architectures, and finally makes PT_RECLAIM depend on 64BIT. This way, PT_RECLAIM can be enabled by default on all 64-bit architectures.
BTW, PT_RECLAIM works well on all 32-bit architectures as well. Although the benefit isn't significant, there's still memory that can be reclaimed. Perhaps PT_RECLAIM can be enabled on all 32-bit architectures in the future.
Comments and suggestions are welcome!
Thanks,
Qi
Qi Zheng (7):
mm: change mm/pt_reclaim.c to use asm/tlb.h instead of
asm-generic/tlb.h
alpha: mm: enable MMU_GATHER_RCU_TABLE_FREE
LoongArch: mm: enable MMU_GATHER_RCU_TABLE_FREE
mips: mm: enable MMU_GATHER_RCU_TABLE_FREE
parisc: mm: enable MMU_GATHER_RCU_TABLE_FREE
um: mm: enable MMU_GATHER_RCU_TABLE_FREE
mm: enable PT_RECLAIM on all 64-bit architectures
arch/alpha/Kconfig | 1 + arch/alpha/include/asm/tlb.h | 6 +++--- arch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/pgalloc.h | 7 +++---- arch/mips/Kconfig | 1 + arch/mips/include/asm/pgalloc.h | 7 +++---- arch/parisc/Kconfig | 1 + arch/parisc/include/asm/tlb.h | 4 ++-- arch/um/Kconfig | 1 + arch/x86/Kconfig | 1 - mm/Kconfig | 9 ++------- mm/pt_reclaim.c | 2 +- 12 files changed, 19 insertions(+), 22 deletions(-)
Signature · 1 line
2.20.1
PATCH v2 1/7 mm: change mm/pt_reclaim.c to use asm/tlb.h instead of asm-generic/tlb.h
From: Qi Zheng <[email protected]>
Generally, the asm/tlb.h will include asm-generic/tlb.h, so change mm/pt_reclaim.c to use asm/tlb.h instead of asm-generic/tlb.h. This can also fix compilation errors on some architecture when CONFIG_PT_RECLAIM is enabled (such as alpha).
--- mm/pt_reclaim.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/pt_reclaim.c +1 -1
@@ -2,7 +2,7 @@#include <linux/hugetlb.h> #include <linux/pgalloc.h> -#include <asm-generic/tlb.h> +#include <asm/tlb.h> #include "internal.h"
Signature · 1 line
2.20.1
PATCH v2 2/7 alpha: mm: enable MMU_GATHER_RCU_TABLE_FREE
In reply to Qi ZhengFrom: Qi Zheng <[email protected]>
On a 64-bit system, madvise(MADV_DONTNEED) may cause a large number of empty PTE page table pages (such as 100GB+). To resolve this problem, first enable MMU_GATHER_RCU_TABLE_FREE to prepare for enabling the PT_RECLAIM feature, which resolves this problem.
--- arch/alpha/Kconfig | 1 + arch/alpha/include/asm/tlb.h | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-)
arch/alpha/Kconfig +1 -0
@@ -38,6 +38,7 @@ config ALPHA select OLD_SIGSUSPEND select CPU_NO_EFFICIENT_FFS if !ALPHA_EV67 select MMU_GATHER_NO_RANGE + select MMU_GATHER_RCU_TABLE_FREE select SPARSEMEM_EXTREME if SPARSEMEM select ZONE_DMA help
arch/alpha/include/asm/tlb.h +3 -3
@@ -4,7 +4,7 @@#include <asm-generic/tlb.h> -#define __pte_free_tlb (tlb, pte, address) pte_free((tlb)->mm, pte) -#define __pmd_free_tlb (tlb, pmd, address) pmd_free((tlb)->mm, pmd) - +#define __pte_free_tlb (tlb, pte, address) tlb_remove_ptdesc((tlb), page_ptdesc(pte)) +#define __pmd_free_tlb (tlb, pmd, address) tlb_remove_ptdesc((tlb), virt_to_ptdesc(pmd)) +#endif
Signature · 1 line
2.20.1
PATCH v2 3/7 LoongArch: mm: enable MMU_GATHER_RCU_TABLE_FREE
In reply to Qi ZhengFrom: Qi Zheng <[email protected]>
On a 64-bit system, madvise(MADV_DONTNEED) may cause a large number of empty PTE page table pages (such as 100GB+). To resolve this problem, first enable MMU_GATHER_RCU_TABLE_FREE to prepare for enabling the PT_RECLAIM feature, which resolves this problem.
--- arch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/pgalloc.h | 7 +++---- 2 files changed, 4 insertions(+), 4 deletions(-)
arch/loongarch/Kconfig +1 -0
@@ -186,6 +186,7 @@ config LOONGARCH select IRQ_LOONGARCH_CPU select LOCK_MM_AND_FIND_VMA select MMU_GATHER_MERGE_VMAS if MMU + select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_RELA if MODULES select NEED_PER_CPU_EMBED_FIRST_CHUNK select NEED_PER_CPU_PAGE_FIRST_CHUNK
arch/loongarch/include/asm/pgalloc.h +3 -4
@@ -55,8 +55,7 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm)return pte; } -#define __pte_free_tlb (tlb, pte, address) \ - tlb_remove_ptdesc((tlb), page_ptdesc(pte)) +#define __pte_free_tlb (tlb, pte, address) tlb_remove_ptdesc((tlb), page_ptdesc(pte))#ifndef __PAGETABLE_PMD_FOLDED
@@ -79,7 +78,7 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) return pmd; } -#define __pmd_free_tlb (tlb, x, addr) pmd_free((tlb)->mm, x) +#define __pmd_free_tlb (tlb, x, addr) tlb_remove_ptdesc((tlb), virt_to_ptdesc(x))#endif
@@ -99,7 +98,7 @@ static inline pud_t *pud_alloc_one(struct mm_struct *mm, unsigned long address) return pud; } -#define __pud_free_tlb (tlb, x, addr) pud_free((tlb)->mm, x) +#define __pud_free_tlb (tlb, x, addr) tlb_remove_ptdesc((tlb), virt_to_ptdesc(x))#endif /* __PAGETABLE_PUD_FOLDED */
Signature · 1 line
2.20.1
PATCH v2 4/7 mips: mm: enable MMU_GATHER_RCU_TABLE_FREE
In reply to Qi ZhengFrom: Qi Zheng <[email protected]>
On a 64-bit system, madvise(MADV_DONTNEED) may cause a large number of empty PTE page table pages (such as 100GB+). To resolve this problem, first enable MMU_GATHER_RCU_TABLE_FREE to prepare for enabling the PT_RECLAIM feature, which resolves this problem.
--- arch/mips/Kconfig | 1 + arch/mips/include/asm/pgalloc.h | 7 +++---- 2 files changed, 4 insertions(+), 4 deletions(-)
arch/mips/Kconfig +1 -0
@@ -99,6 +99,7 @@ config MIPS select IRQ_FORCED_THREADING select ISA if EISA select LOCK_MM_AND_FIND_VMA + select MMU_GATHER_RCU_TABLE_FREE select MODULES_USE_ELF_REL if MODULES select MODULES_USE_ELF_RELA if MODULES && 64BIT select PERF_USE_VMALLOC
arch/mips/include/asm/pgalloc.h +3 -4
@@ -48,8 +48,7 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)extern void pgd_init (void *addr);extern pgd_t *pgd_alloc (struct mm_struct *mm); -#define __pte_free_tlb (tlb, pte, address) \ - tlb_remove_ptdesc((tlb), page_ptdesc(pte)) +#define __pte_free_tlb (tlb, pte, address) tlb_remove_ptdesc((tlb), page_ptdesc(pte))#ifndef __PAGETABLE_PMD_FOLDED
@@ -72,7 +71,7 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) return pmd; } -#define __pmd_free_tlb (tlb, x, addr) pmd_free((tlb)->mm, x) +#define __pmd_free_tlb (tlb, x, addr) tlb_remove_ptdesc((tlb), virt_to_ptdesc(x))#endif
@@ -98,7 +97,7 @@ static inline void p4d_populate(struct mm_struct *mm, p4d_t *p4d, pud_t *pud) set_p4d(p4d, __p4d((unsigned long)pud)); } -#define __pud_free_tlb (tlb, x, addr) pud_free((tlb)->mm, x) +#define __pud_free_tlb (tlb, x, addr) tlb_remove_ptdesc((tlb), virt_to_ptdesc(x))#endif /* __PAGETABLE_PUD_FOLDED */
Signature · 1 line
2.20.1
PATCH v2 5/7 parisc: mm: enable MMU_GATHER_RCU_TABLE_FREE
In reply to Qi ZhengFrom: Qi Zheng <[email protected]>
On a 64-bit system, madvise(MADV_DONTNEED) may cause a large number of empty PTE page table pages (such as 100GB+). To resolve this problem, first enable MMU_GATHER_RCU_TABLE_FREE to prepare for enabling the PT_RECLAIM feature, which resolves this problem.
--- arch/parisc/Kconfig | 1 + arch/parisc/include/asm/tlb.h | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-)
arch/parisc/Kconfig +1 -0
@@ -79,6 +79,7 @@ config PARISC select GENERIC_CLOCKEVENTS select CPU_NO_EFFICIENT_FFS select THREAD_INFO_IN_TASK + select MMU_GATHER_RCU_TABLE_FREE select NEED_DMA_MAP_STATE select NEED_SG_DMA_LENGTH select HAVE_ARCH_KGDB
arch/parisc/include/asm/tlb.h +2 -2
@@ -5,8 +5,8 @@#include <asm-generic/tlb.h> #if CONFIG_PGTABLE_LEVELS ==3 -#define __pmd_free_tlb (tlb, pmd, addr) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb (tlb, pmd, addr) tlb_remove_ptdesc((tlb), virt_to_ptdesc(pmd))#endif -#define __pte_free_tlb (tlb, pte, addr) pte_free((tlb)->mm, pte) +#define __pte_free_tlb (tlb, pte, addr) tlb_remove_ptdesc((tlb), page_ptdesc(pte))#endif
Signature · 1 line
2.20.1
PATCH v2 6/7 um: mm: enable MMU_GATHER_RCU_TABLE_FREE
In reply to Qi ZhengFrom: Qi Zheng <[email protected]>
On a 64-bit system, madvise(MADV_DONTNEED) may cause a large number of empty PTE page table pages (such as 100GB+). To resolve this problem, first enable MMU_GATHER_RCU_TABLE_FREE to prepare for enabling the PT_RECLAIM feature, which resolves this problem.
--- arch/um/Kconfig | 1 + 1 file changed, 1 insertion(+)
arch/um/Kconfig +1 -0
@@ -41,6 +41,7 @@ config UML select HAVE_SYSCALL_TRACEPOINTS select THREAD_INFO_IN_TASK select SPARSE_IRQ + select MMU_GATHER_RCU_TABLE_FREE config MMU bool
Signature · 1 line
2.20.1
PATCH v2 7/7 mm: enable PT_RECLAIM on all 64-bit architectures
In reply to Qi ZhengFrom: Qi Zheng <[email protected]>
Now, the MMU_GATHER_RCU_TABLE_FREE is enabled on all 64-bit architectures, so make PT_RECLAIM depend on 64BIT, thereby enabling PT_RECLAIM on all 64-bit architectures.
--- arch/x86/Kconfig | 1 - mm/Kconfig | 9 ++------- 2 files changed, 2 insertions(+), 8 deletions(-)
arch/x86/Kconfig +0 -1
@@ -330,7 +330,6 @@ config X86 select FUNCTION_ALIGNMENT_4B imply IMA_SECURE_AND_OR_TRUSTED_BOOT if EFI select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE - select ARCH_SUPPORTS_PT_RECLAIM if X86_64 select ARCH_SUPPORTS_SCHED_SMT if SMP select SCHED_SMT if SMP select ARCH_SUPPORTS_SCHED_CLUSTER if SMP
mm/Kconfig +2 -7
@@ -1448,14 +1448,9 @@ config ARCH_HAS_USER_SHADOW_STACK The architecture has hardware support for userspace shadow call stacks (eg, x86 CET, arm64 GCS or RISC-V Zicfiss). -config ARCH_SUPPORTS_PT_RECLAIM - def_bool n - config PT_RECLAIM - bool "reclaim empty user page table pages" - default y - depends on ARCH_SUPPORTS_PT_RECLAIM && MMU && SMP - select MMU_GATHER_RCU_TABLE_FREE + def_bool y + depends on 64BIT help Try to reclaim empty user page table pages in paths other than munmap and exit_mmap path.
Signature · 1 line
2.20.1
David Hildenbrand (Red Hat)
maintainerRe: [PATCH v2 7/7] mm: enable PT_RECLAIM on all 64-bit architectures
Quoted 32 lines · Qi Zheng
From: Qi Zheng <[email protected]>
Now, the
MMU_GATHER_RCU_TABLE_FREEis enabled on all 64-bit architectures, so makePT_RECLAIMdepend on 64BIT, thereby enablingPT_RECLAIMon all 64-bit architectures.Signed-off-by Qi Zheng <[email protected]>--- arch/x86/Kconfig | 1 - mm/Kconfig | 9 ++------- 2 files changed, 2 insertions(+), 8 deletions(-)arch/x86/Kconfig +0
-1@@ -330,7 +330,6 @@ config X86 select FUNCTION_ALIGNMENT_4B imply IMA_SECURE_AND_OR_TRUSTED_BOOT if EFI select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE - select ARCH_SUPPORTS_PT_RECLAIM if X86_64 select ARCH_SUPPORTS_SCHED_SMT if SMP select SCHED_SMT if SMP select ARCH_SUPPORTS_SCHED_CLUSTER if SMPmm/Kconfig +0
-0@@ -1448,14 +1448,9 @@ config ARCH_HAS_USER_SHADOW_STACK The architecture has hardware support for userspace shadow call stacks (eg, x86 CET, arm64 GCS or RISC-V Zicfiss).
mm/Kconfig config ARCH_HAS_USER_SHADOW_STACK · line 1453
Quoted 10 lines
mm/Kconfig +2
-7-config ARCH_SUPPORTS_PT_RECLAIM - def_bool n - config PT_RECLAIM - bool "reclaim empty user page table pages" - default y - depends on ARCH_SUPPORTS_PT_RECLAIM && MMU && SMP - select MMU_GATHER_RCU_TABLE_FREE + def_bool y + depends on 64BIT
As discussed in the other thread, likely
config PT_RECLAIM
def_bool y depends on MMU_GATHER_RCU_TABLE_FREE && 64BIT
Could be nice, and if possible even dropping the 64BIT limitation as well if there is no need to.
Signature · 3 lines
Cheers David
Re: [PATCH v2 1/7] mm: change mm/pt_reclaim.c to use asm/tlb.h instead of asm-generic/tlb.h
In reply to Qi ZhengQuoted 6 lines · Qi Zheng
From: Qi Zheng <[email protected]>
Generally, the
asm/tlb.hwill includeasm-generic/tlb.h, so changemm/pt_reclaim.cto useasm/tlb.hinstead ofasm-generic/tlb.h. This can also fix compilation errors on some architecture whenCONFIG_PT_RECLAIMis enabled (such as alpha).
"This is a preparation for enabling CONFIG_PT_RECLAIM on other architectures, such as alpha."
Quoted 11 lines
Signed-off-by Qi Zheng <[email protected]>--- mm/pt_reclaim.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/pt_reclaim.c +0
-0
mm/pt_reclaim.c line 8
Quoted 8 lines
mm/pt_reclaim.c +1
-1@@ -2,7 +2,7 @@#include <linux/hugetlb.h> #include <linux/pgalloc.h> -#include <asm-generic/tlb.h> +#include <asm/tlb.h> #include "internal.h"
Right, we're using pte_free_tlb(), and the default lives in include/asm-generic/tlb.h.
Signature · 3 lines
Cheers David
Re: [PATCH v2 7/7] mm: enable PT_RECLAIM on all 64-bit architectures
In reply to David Hildenbrand (Red Hat)Quoted 45 lines · David Hildenbrand (Red Hat)
Quoted 34 lines · Qi Zheng
From: Qi Zheng <[email protected]>
Now, the
MMU_GATHER_RCU_TABLE_FREEis enabled on all 64-bit architectures, so makePT_RECLAIMdepend on 64BIT, thereby enablingPT_RECLAIMon all 64-bit architectures.Signed-off-by Qi Zheng <[email protected]>---
arch/x86/Kconfig| 1 -
mm/Kconfig | 9 ++-------2 files changed, 2 insertions(+), 8 deletions(-)arch/x86/Kconfig +0
-0@@ -330,7 +330,6 @@ config X86select
FUNCTION_ALIGNMENT_4B
imply IMA_SECURE_AND_OR_TRUSTED_BOOT if EFI selectHAVE_DYNAMIC_FTRACE_NO_PATCHABLE- select ARCH_SUPPORTS_PT_RECLAIM ifX86_64select ARCH_SUPPORTS_SCHED_SMT if SMP select SCHED_SMT if SMP select ARCH_SUPPORTS_SCHED_CLUSTER if SMPmm/Kconfig +0
-0@@ -1448,14 +1448,9 @@ config ARCH_HAS_USER_SHADOW_STACKThe architecture has hardware support for userspace shadow call stacks (eg, x86 CET, arm64 GCS or RISC-V Zicfiss).
mm/Kconfig +0
-3-config ARCH_SUPPORTS_PT_RECLAIM - def_bool n -Quoted 7 lines
config
PT_RECLAIM
- bool "reclaim empty user page table pages" - default y
- depends onARCH_SUPPORTS_PT_RECLAIM&& MMU && SMP - selectMMU_GATHER_RCU_TABLE_FREE
+def_booly
+ depends on 64BITAs discussed in the other thread, likely
Quoted 5 lines
config
PT_RECLAIM
def_bool y
depends onMMU_GATHER_RCU_TABLE_FREE&& 64BITCould be nice, and if possible even dropping the 64BIT limitation as well if there is no need to.
I think it's ok to drop the 64BIT limitation. There should be some 32-bit architectures that already enable MMU_GATHER_RCU_TABLE_FREE.
Quoted 2 lines
Re: [PATCH v2 1/7] mm: change mm/pt_reclaim.c to use asm/tlb.h instead of asm-generic/tlb.h
In reply to David Hildenbrand (Red Hat)Quoted 9 lines · David Hildenbrand (Red Hat)
Quoted 6 lines · Qi Zheng
From: Qi Zheng <[email protected]>
Generally, the
asm/tlb.hwill includeasm-generic/tlb.h, so changemm/pt_reclaim.cto useasm/tlb.hinstead ofasm-generic/tlb.h. This can also fix compilation errors on some architecture whenCONFIG_PT_RECLAIMis enabled (such as alpha)."This is a preparation for enabling
CONFIG_PT_RECLAIMon other architectures, such as alpha."
OK, will modify it in the next version.
Quoted 20 lines
Quoted 12 lines
Signed-off-by Qi Zheng <[email protected]>---
mm/pt_reclaim.c| 2 +-1 file changed, 1 insertion(+), 1 deletion(-)mm/pt_reclaim.c +0
-0mm/pt_reclaim.c line 5
Quoted 5 lines
mm/pt_reclaim.c +1
-1@@ -2,7 +2,7 @@ #include <linux/hugetlb.h> #include <linux/pgalloc.h> -#include <asm-generic/tlb.h> +#include <asm/tlb.h> #include "internal.h"Right, we're using
pte_free_tlb(), and the default lives ininclude/asm-generic/tlb.h.
Quoted 1 line
Thanks!