Skip to contentSkip to conversation

mm/debug_vm_pgtable fixes

PATCH Superseded Aneesh Kumar K.V 4 participants 33 messages Aug 19, 2020 – Sep 1, 2020

Subsystems (5)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover mm/debug_vm_pgtable fixes

This patch series includes fixes for debug_vm_pgtable test code so that they follow page table updates rules correctly. The first two patches introduce changes w.r.t ppc64. The patches are included in this series for completeness. We can merge them via ppc64 tree if required.

Hugetlb test is disabled on ppc64 because that needs larger change to satisfy page table update rules.

Changes from V1:
* Address review feedback
* drop test specific pfn_pte and pfn_pmd.
* Update ppc64 page table helper to add _PAGE_PTE

Aneesh Kumar K.V (13):
powerpc/mm: Add DEBUG_VM WARN for pmd_clear
powerpc/mm: Move setting pte specific flags to pfn_pte mm/debug_vm_pgtable/ppc64: Avoid setting top bits in radom value mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge
vmap support.
mm/debug_vm_pgtable/savedwrite: Enable savedwrite test with
CONFIG_NUMA_BALANCING
mm/debug_vm_pgtable/THP: Mark the pte entry huge before using
set_pmd/pud_at
mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an
existing pte entry
mm/debug_vm_pgtable/thp: Use page table depost/withdraw with THP mm/debug_vm_pgtable/locks: Move non page table modifying test together mm/debug_vm_pgtable/locks: Take correct page table lock mm/debug_vm_pgtable/pmd_clear: Don't use pmd/pud_clear on pte entries mm/debug_vm_pgtable/hugetlb: Disable hugetlb test on ppc64 mm/debug_vm_pgtable: populate a pte entry before fetching it

 arch/powerpc/include/asm/book3s/64/pgtable.h |  29 +++-
 arch/powerpc/include/asm/nohash/pgtable.h    |   5 -
 arch/powerpc/mm/book3s64/pgtable.c           |   2 +-
 arch/powerpc/mm/pgtable.c                    |   5 -
 include/linux/io.h                           |  12 ++
 mm/debug_vm_pgtable.c                        | 151 +++++++++++--------
 6 files changed, 127 insertions(+), 77 deletions(-)
Signature · 1 line
2.26.2
#1 lore raw

PATCH v2 1/13 powerpc/mm: Add DEBUG_VM WARN for pmd_clear

With the hash page table, the kernel should not use pmd_clear for clearing huge pte entries. Add a DEBUG_VM WARN to catch the wrong usage.

---
 arch/powerpc/include/asm/book3s/64/pgtable.h | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
arch/powerpc/include/asm/book3s/64/pgtable.h +14 -0
index 6de56c3b33c4..079211968987 100644
@@ -868,6 +868,13 @@ static inline bool pte_ci(pte_t pte)
 
 static inline void pmd_clear(pmd_t *pmdp)
 {
+	if (IS_ENABLED(CONFIG_DEBUG_VM) && !radix_enabled()) {
+		/*
+		 * Don't use this if we can possibly have a hash page table
+		 * entry mapping this.
+		 */
+		WARN_ON((pmd_val(*pmdp) & (H_PAGE_HASHPTE | _PAGE_PTE)) == (H_PAGE_HASHPTE | _PAGE_PTE));
+	}
 	*pmdp = __pmd(0);
 }
 
@@ -916,6 +923,13 @@ static inline int pmd_bad(pmd_t pmd)
 
 static inline void pud_clear(pud_t *pudp)
 {
+	if (IS_ENABLED(CONFIG_DEBUG_VM) && !radix_enabled()) {
+		/*
+		 * Don't use this if we can possibly have a hash page table
+		 * entry mapping this.
+		 */
+		WARN_ON((pud_val(*pudp) & (H_PAGE_HASHPTE | _PAGE_PTE)) == (H_PAGE_HASHPTE | _PAGE_PTE));
+	}
 	*pudp = __pud(0);
 }
 
Signature · 1 line
2.26.2
#2 lore raw

PATCH v2 2/13 powerpc/mm: Move setting pte specific flags to pfn_pte

In reply to Aneesh Kumar K.V

powerpc used to set the pte specific flags in set_pte_at(). This is different from other architectures. To be consistent with other architecture update pfn_pte and pfn_pmd to set _PAGE_PTE on ppc64. Also, drop now unused pte_mkpte.

We add a VM_WARN_ON() to catch the usage of calling set_pte_at() without setting _PAGE_PTE bit. We will remove that after a few releases.

---
 arch/powerpc/include/asm/book3s/64/pgtable.h | 15 +++++++++------
 arch/powerpc/include/asm/nohash/pgtable.h    |  5 -----
 arch/powerpc/mm/book3s64/pgtable.c           |  2 +-
 arch/powerpc/mm/pgtable.c                    |  5 -----
 4 files changed, 10 insertions(+), 17 deletions(-)
arch/powerpc/include/asm/book3s/64/pgtable.h +9 -6
index 079211968987..2382fd516f6b 100644
@@ -619,7 +619,7 @@ static inline pte_t pfn_pte(unsigned long pfn, pgprot_t pgprot)
 	VM_BUG_ON(pfn >> (64 - PAGE_SHIFT));
 	VM_BUG_ON((pfn << PAGE_SHIFT) & ~PTE_RPN_MASK);
 
-	return __pte(((pte_basic_t)pfn << PAGE_SHIFT) | pgprot_val(pgprot));
+	return __pte(((pte_basic_t)pfn << PAGE_SHIFT) | pgprot_val(pgprot) | _PAGE_PTE);
 }
 
 static inline unsigned long pte_pfn(pte_t pte)
@@ -655,11 +655,6 @@ static inline pte_t pte_mkexec(pte_t pte)
 	return __pte_raw(pte_raw(pte) | cpu_to_be64(_PAGE_EXEC));
 }
 
-static inline pte_t pte_mkpte(pte_t pte)
-{
-	return __pte_raw(pte_raw(pte) | cpu_to_be64(_PAGE_PTE));
-}
-
 static inline pte_t pte_mkwrite(pte_t pte)
 {
 	/*
@@ -823,6 +818,14 @@ static inline int pte_none(pte_t pte)
 static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr,
 				pte_t *ptep, pte_t pte, int percpu)
 {
+
+	VM_WARN_ON(!(pte_raw(pte) & cpu_to_be64(_PAGE_PTE)));
+	/*
+	 * Keep the _PAGE_PTE added till we are sure we handle _PAGE_PTE
+	 * in all the callers.
+	 */
+	 pte = __pte_raw(pte_raw(pte) | cpu_to_be64(_PAGE_PTE));
+
 	if (radix_enabled())
 		return radix__set_pte_at(mm, addr, ptep, pte, percpu);
 	return hash__set_pte_at(mm, addr, ptep, pte, percpu);
arch/powerpc/include/asm/nohash/pgtable.h +0 -5
index 4b7c3472eab1..6277e7596ae5 100644
@@ -140,11 +140,6 @@ static inline pte_t pte_mkold(pte_t pte)
 	return __pte(pte_val(pte) & ~_PAGE_ACCESSED);
 }
 
-static inline pte_t pte_mkpte(pte_t pte)
-{
-	return pte;
-}
-
 static inline pte_t pte_mkspecial(pte_t pte)
 {
 	return __pte(pte_val(pte) | _PAGE_SPECIAL);
arch/powerpc/mm/book3s64/pgtable.c +1 -1
index e18ae50a275c..3b4da7c63e28 100644
@@ -141,7 +141,7 @@ pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot)
 	unsigned long pmdv;
 
 	pmdv = (pfn << PAGE_SHIFT) & PTE_RPN_MASK;
-	return pmd_set_protbits(__pmd(pmdv), pgprot);
+	return __pmd(pmdv | pgprot_val(pgprot) | _PAGE_PTE);
 }
 
 pmd_t mk_pmd(struct page *page, pgprot_t pgprot)
arch/powerpc/mm/pgtable.c +0 -5
index 9c0547d77af3..ab57b07ef39a 100644
@@ -184,9 +184,6 @@ void set_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep,
 	 */
 	VM_WARN_ON(pte_hw_valid(*ptep) && !pte_protnone(*ptep));
 
-	/* Add the pte bit when trying to set a pte */
-	pte = pte_mkpte(pte);
-
 	/* Note: mm->context.id might not yet have been assigned as
 	 * this context might not have been activated yet when this
 	 * is called.
@@ -275,8 +272,6 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_
 	 */
 	VM_WARN_ON(pte_hw_valid(*ptep) && !pte_protnone(*ptep));
 
-	pte = pte_mkpte(pte);
-
 	pte = set_pte_filter(pte);
 
 	val = pte_val(pte);
Signature · 1 line
2.26.2
#3 lore raw

PATCH v2 3/13 mm/debug_vm_pgtable/ppc64: Avoid setting top bits in radom value

In reply to Aneesh Kumar K.V

ppc64 use bit 62 to indicate a pte entry (_PAGE_PTE). Avoid setting that bit in random value.

---
 mm/debug_vm_pgtable.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
mm/debug_vm_pgtable.c +6 -3
index 086309fb9b6f..57259e2dbd17 100644
@@ -44,10 +44,13 @@
  * entry type. But these bits might affect the ability to clear entries with
  * pxx_clear() because of how dynamic page table folding works on s390. So
  * while loading up the entries do not change the lower 4 bits. It does not
- * have affect any other platform.
+ * have affect any other platform. Also avoid the 62nd bit on ppc64 that is
+ * used to mark a pte entry.
  */
-#define S390_MASK_BITS	4
-#define RANDOM_ORVALUE	GENMASK(BITS_PER_LONG - 1, S390_MASK_BITS)
+#define S390_SKIP_MASK		GENMASK(3, 0)
+#define PPC64_SKIP_MASK		GENMASK(62, 62)
+#define ARCH_SKIP_MASK (S390_SKIP_MASK | PPC64_SKIP_MASK)
+#define RANDOM_ORVALUE (GENMASK(BITS_PER_LONG - 1, 0) & ~ARCH_SKIP_MASK)
 #define RANDOM_NZVALUE	GENMASK(7, 0)
 
 static void __init pte_basic_tests(unsigned long pfn, pgprot_t prot)
Signature · 1 line
2.26.2
#4 lore raw

PATCH v2 4/13 mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge vmap support.

In reply to Aneesh Kumar K.V

ppc64 supports huge vmap only with radix translation. Hence use arch helper to determine the huge vmap support.

---
 include/linux/io.h    | 12 ++++++++++++
 mm/debug_vm_pgtable.c |  4 ++--
 2 files changed, 14 insertions(+), 2 deletions(-)
include/linux/io.h +12 -0
index 8394c56babc2..0b1ecda0cc86 100644
@@ -38,6 +38,18 @@ int arch_ioremap_pud_supported(void);
 int arch_ioremap_pmd_supported(void);
 #else
 static inline void ioremap_huge_init(void) { }
+static inline int arch_ioremap_p4d_supported(void)
+{
+	return false;
+}
+static inline int arch_ioremap_pud_supported(void)
+{
+	return false;
+}
+static inline int arch_ioremap_pmd_supported(void)
+{
+	return false;
+}
 #endif
 
 /*
mm/debug_vm_pgtable.c +2 -2
index 57259e2dbd17..cf3c4792b4a2 100644
@@ -206,7 +206,7 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
 {
 	pmd_t pmd;
 
-	if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
+	if (!arch_ioremap_pmd_supported())
 		return;
 
 	pr_debug("Validating PMD huge\n");
@@ -320,7 +320,7 @@ static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot)
 {
 	pud_t pud;
 
-	if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
+	if (!arch_ioremap_pud_supported())
 		return;
 
 	pr_debug("Validating PUD huge\n");
Signature · 1 line
2.26.2
#5 lore raw

Re: [PATCH v2 04/13] mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge vmap support.

I love your patch! Yet something to improve:

Hi "Aneesh,

I love your patch! Yet something to improve:

[auto build test ERROR on hnaz-linux-mm/master]
[also build test ERROR on powerpc/next linus/master v5.9-rc1 next-20200819] [cannot apply to mmotm/master]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Aneesh-Kumar-K-V/mm-debug_vm_pgtable-fixes/20200819-213446
base:   https://github.com/hnaz/linux-mm master
config: i386-randconfig-s002-20200818 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
reproduce:
        # apt-get install sparse
        # sparse version: v0.6.2-183-gaa6ede3b-dirty
        # save the attached .config to linux build tree
        make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' ARCH=i386 

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

>> arch/x86/mm/ioremap.c:484:12: error: redefinition of 'arch_ioremap_p4d_supported'
     484 | int __init arch_ioremap_p4d_supported(void)
         |            ^~~~~~~~~~~~~~~~~~~~~~~~~~
   In file included from arch/x86/mm/ioremap.c:12:
   include/linux/io.h:41:19: note: previous definition of 'arch_ioremap_p4d_supported' was here
      41 | static inline int arch_ioremap_p4d_supported(void)
         |                   ^~~~~~~~~~~~~~~~~~~~~~~~~~
>> arch/x86/mm/ioremap.c:489:12: error: redefinition of 'arch_ioremap_pud_supported'
     489 | int __init arch_ioremap_pud_supported(void)
         |            ^~~~~~~~~~~~~~~~~~~~~~~~~~
   In file included from arch/x86/mm/ioremap.c:12:
   include/linux/io.h:45:19: note: previous definition of 'arch_ioremap_pud_supported' was here
      45 | static inline int arch_ioremap_pud_supported(void)
         |                   ^~~~~~~~~~~~~~~~~~~~~~~~~~
>> arch/x86/mm/ioremap.c:498:12: error: redefinition of 'arch_ioremap_pmd_supported'

498 | int __init arch_ioremap_pmd_supported(void)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from arch/x86/mm/ioremap.c:12:
include/linux/io.h:49:19: note: previous definition of 'arch_ioremap_pmd_supported' was here
49 | static inline int arch_ioremap_pmd_supported(void)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
arch/x86/mm/ioremap.c:737:17: warning: no previous prototype for 'early_memremap_pgprot_adjust' [-Wmissing-prototypes]
737 | pgprot_t __init early_memremap_pgprot_adjust(resource_size_t phys_addr,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~

# https://github.com/0day-ci/linux/commit/260b675444a7d5afa9ccef47ca9e588fb18d01a3
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Aneesh-Kumar-K-V/mm-debug_vm_pgtable-fixes/20200819-213446 git checkout 260b675444a7d5afa9ccef47ca9e588fb18d01a3
vim +/arch_ioremap_p4d_supported +484 arch/x86/mm/ioremap.c

^1da177e4c3f41 arch/i386/mm/ioremap.c Linus Torvalds 2005-04-16 483

0f472d04f59ff8 arch/x86/mm/ioremap.c  Anshuman Khandual 2019-07-16 @484  int __init arch_ioremap_p4d_supported(void)
0f472d04f59ff8 arch/x86/mm/ioremap.c  Anshuman Khandual 2019-07-16  485  {
0f472d04f59ff8 arch/x86/mm/ioremap.c  Anshuman Khandual 2019-07-16  486  	return 0;
0f472d04f59ff8 arch/x86/mm/ioremap.c  Anshuman Khandual 2019-07-16  487  }
0f472d04f59ff8 arch/x86/mm/ioremap.c  Anshuman Khandual 2019-07-16  488  
1e6277de3a2337 arch/x86/mm/ioremap.c  Jan Beulich       2015-05-28 @489  int __init arch_ioremap_pud_supported(void)
5d72b4fba40ef4 arch/x86/mm/ioremap.c  Toshi Kani        2015-04-14  490  {
5d72b4fba40ef4 arch/x86/mm/ioremap.c  Toshi Kani        2015-04-14  491  #ifdef CONFIG_X86_64
b8291adc191abe arch/x86/mm/ioremap.c  Borislav Petkov   2016-03-29  492  	return boot_cpu_has(X86_FEATURE_GBPAGES);
5d72b4fba40ef4 arch/x86/mm/ioremap.c  Toshi Kani        2015-04-14  493  #else
5d72b4fba40ef4 arch/x86/mm/ioremap.c  Toshi Kani        2015-04-14  494  	return 0;
5d72b4fba40ef4 arch/x86/mm/ioremap.c  Toshi Kani        2015-04-14  495  #endif
5d72b4fba40ef4 arch/x86/mm/ioremap.c  Toshi Kani        2015-04-14  496  }
5d72b4fba40ef4 arch/x86/mm/ioremap.c  Toshi Kani        2015-04-14  497  
1e6277de3a2337 arch/x86/mm/ioremap.c  Jan Beulich       2015-05-28 @498  int __init arch_ioremap_pmd_supported(void)
5d72b4fba40ef4 arch/x86/mm/ioremap.c  Toshi Kani        2015-04-14  499  {
16bf92261b1b6c arch/x86/mm/ioremap.c  Borislav Petkov   2016-03-29  500  	return boot_cpu_has(X86_FEATURE_PSE);
5d72b4fba40ef4 arch/x86/mm/ioremap.c  Toshi Kani        2015-04-14  501  }
5d72b4fba40ef4 arch/x86/mm/ioremap.c  Toshi Kani        2015-04-14  502  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#6 lore raw

Re: [PATCH v2 04/13] mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge vmap support.

Quoted 26 lines · kernel test robot

Hi "Aneesh,

I love your patch! Yet something to improve:

[auto build test ERROR on hnaz-linux-mm/master]
[also build test ERROR on powerpc/next linus/master v5.9-rc1 next-20200819] [cannot apply to mmotm/master]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Aneesh-Kumar-K-V/mm-debug_vm_pgtable-fixes/20200819-213446
base:   https://github.com/hnaz/linux-mm master
config: i386-randconfig-s002-20200818 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
reproduce:
        # apt-get install sparse
        # sparse version: v0.6.2-183-gaa6ede3b-dirty
        # save the attached .config to linux build tree
        make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' ARCH=i386 

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

Quoted 7 lines
Quoted 1 line
Quoted 1 line

arch/x86/mm/ioremap.c:484:12: error: redefinition of 'arch_ioremap_p4d_supported'

484 | int __init arch_ioremap_p4d_supported(void)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from arch/x86/mm/ioremap.c:12:
include/linux/io.h:41:19: note: previous definition of 'arch_ioremap_p4d_supported' was here
41 | static inline int arch_ioremap_p4d_supported(void)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~

I guess trying to work this out without using #ifdef is complex. I ended up with the below

1 file changed, 12 insertions(+), 2 deletions(-)

mm/debug_vm_pgtable.c | 14 ++++++++++++--

modified mm/debug_vm_pgtable.c
@@ -202,11 +202,12 @@ static void __init pmd_leaf_tests(unsigned long pfn, pgprot_t prot)
WARN_ON(!pmd_leaf(pmd));
}

+#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP
static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) {
pmd_t pmd;

- if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
+ if (!arch_ioremap_pmd_supported())
return;

pr_debug("Validating PMD huge\n");
@@ -220,6 +221,10 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
pmd = READ_ONCE(*pmdp);
WARN_ON(!pmd_none(pmd));
}
+#else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */
+static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) { } +#endif /* !CONFIG_HAVE_ARCH_HUGE_VMAP */
+

static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot) {
@@ -316,11 +321,12 @@ static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot)
WARN_ON(!pud_leaf(pud));
}

+#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP
static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) {
pud_t pud;

- if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
+ if (!arch_ioremap_pud_supported())
return;

pr_debug("Validating PUD huge\n");
@@ -334,6 +340,10 @@ static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot)
pud = READ_ONCE(*pudp);
WARN_ON(!pud_none(pud));
}
+#else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */
+static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) { } +#endif /* !CONFIG_HAVE_ARCH_HUGE_VMAP */
+
#else /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */
static void __init pud_basic_tests(unsigned long pfn, pgprot_t prot) { }
static void __init pud_advanced_tests(struct mm_struct *mm,

#7 lore raw

Re: [PATCH v2 04/13] mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge vmap support.

In reply to Aneesh Kumar K.V

mm/debug_vm_pgtable.c

Quoted 36 lines · Aneesh Kumar K.V

ppc64 supports huge vmap only with radix translation. Hence use arch helper to determine the huge vmap support.

---
 include/linux/io.h    | 12 ++++++++++++
 mm/debug_vm_pgtable.c |  4 ++--
 2 files changed, 14 insertions(+), 2 deletions(-)
include/linux/io.h +12 -0
index 8394c56babc2..0b1ecda0cc86 100644
@@ -38,6 +38,18 @@ int arch_ioremap_pud_supported(void);
 int arch_ioremap_pmd_supported(void);
 #else
 static inline void ioremap_huge_init(void) { }
+static inline int arch_ioremap_p4d_supported(void)
+{
+	return false;
+}
+static inline int arch_ioremap_pud_supported(void)
+{
+	return false;
+}
+static inline int arch_ioremap_pmd_supported(void)
+{
+	return false;
+}
 #endif
 
 /*
mm/debug_vm_pgtable.c +0 -0
index 57259e2dbd17..cf3c4792b4a2 100644

This would need an explicit inclusion of <linux/io.h> in order to prevent build failure in some cases.

Quoted 19 lines
mm/debug_vm_pgtable.c +2 -2
@@ -206,7 +206,7 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
 {
 	pmd_t pmd;
 
-	if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
+	if (!arch_ioremap_pmd_supported())
 		return;
 
 	pr_debug("Validating PMD huge\n");
@@ -320,7 +320,7 @@ static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot)
 {
 	pud_t pud;
 
-	if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP))
+	if (!arch_ioremap_pud_supported())
 		return;
 
 	pr_debug("Validating PUD huge\n");
#8 lore raw

PATCH v2 5/13 mm/debug_vm_pgtable/savedwrite: Enable savedwrite test with CONFIG_NUMA_BALANCING

In reply to Aneesh Kumar K.V

Saved write support was added to track the write bit of a pte after marking the pte protnone. This was done so that AUTONUMA can convert a write pte to protnone and still track the old write bit. When converting it back we set the pte write bit correctly thereby avoiding a write fault again. Hence enable the test only when CONFIG_NUMA_BALANCING is enabled and use protnone protflags.

---
 mm/debug_vm_pgtable.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
mm/debug_vm_pgtable.c +9 -2
index cf3c4792b4a2..ffa10ede6842 100644
@@ -114,10 +114,14 @@ static void __init pte_savedwrite_tests(unsigned long pfn, pgprot_t prot)
 {
 	pte_t pte = pfn_pte(pfn, prot);
 
+	if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
+		return;
+
 	pr_debug("Validating PTE saved write\n");
 	WARN_ON(!pte_savedwrite(pte_mk_savedwrite(pte_clear_savedwrite(pte))));
 	WARN_ON(pte_savedwrite(pte_clear_savedwrite(pte_mk_savedwrite(pte))));
 }
+
 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
 static void __init pmd_basic_tests(unsigned long pfn, pgprot_t prot)
 {
@@ -225,6 +229,9 @@ static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
 {
 	pmd_t pmd = pfn_pmd(pfn, prot);
 
+	if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
+		return;
+
 	pr_debug("Validating PMD saved write\n");
 	WARN_ON(!pmd_savedwrite(pmd_mk_savedwrite(pmd_clear_savedwrite(pmd))));
 	WARN_ON(pmd_savedwrite(pmd_clear_savedwrite(pmd_mk_savedwrite(pmd))));
@@ -1005,8 +1012,8 @@ static int __init debug_vm_pgtable(void)
 	pmd_huge_tests(pmdp, pmd_aligned, prot);
 	pud_huge_tests(pudp, pud_aligned, prot);
 
-	pte_savedwrite_tests(pte_aligned, prot);
-	pmd_savedwrite_tests(pmd_aligned, prot);
+	pte_savedwrite_tests(pte_aligned, protnone);
+	pmd_savedwrite_tests(pmd_aligned, protnone);
 
 	pte_unmap_unlock(ptep, ptl);
 
Signature · 1 line
2.26.2
#9 lore raw

PATCH v2 6/13 mm/debug_vm_pgtable/THP: Mark the pte entry huge before using set_pmd/pud_at

In reply to Aneesh Kumar K.V

kernel expects entries to be marked huge before we use set_pmd_at()/set_pud_at().

---
 mm/debug_vm_pgtable.c | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)
mm/debug_vm_pgtable.c +11 -8
index ffa10ede6842..76f4c713e5a3 100644
@@ -150,7 +150,7 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
 				      unsigned long pfn, unsigned long vaddr,
 				      pgprot_t prot)
 {
-	pmd_t pmd = pfn_pmd(pfn, prot);
+	pmd_t pmd;
 
 	if (!has_transparent_hugepage())
 		return;
@@ -159,19 +159,19 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
 	/* Align the address wrt HPAGE_PMD_SIZE */
 	vaddr = (vaddr & HPAGE_PMD_MASK) + HPAGE_PMD_SIZE;
 
-	pmd = pfn_pmd(pfn, prot);
+	pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
 	set_pmd_at(mm, vaddr, pmdp, pmd);
 	pmdp_set_wrprotect(mm, vaddr, pmdp);
 	pmd = READ_ONCE(*pmdp);
 	WARN_ON(pmd_write(pmd));
 
-	pmd = pfn_pmd(pfn, prot);
+	pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
 	set_pmd_at(mm, vaddr, pmdp, pmd);
 	pmdp_huge_get_and_clear(mm, vaddr, pmdp);
 	pmd = READ_ONCE(*pmdp);
 	WARN_ON(!pmd_none(pmd));
 
-	pmd = pfn_pmd(pfn, prot);
+	pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
 	pmd = pmd_wrprotect(pmd);
 	pmd = pmd_mkclean(pmd);
 	set_pmd_at(mm, vaddr, pmdp, pmd);
@@ -267,7 +267,7 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
 				      unsigned long pfn, unsigned long vaddr,
 				      pgprot_t prot)
 {
-	pud_t pud = pfn_pud(pfn, prot);
+	pud_t pud;
 
 	if (!has_transparent_hugepage())
 		return;
@@ -276,25 +276,28 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
 	/* Align the address wrt HPAGE_PUD_SIZE */
 	vaddr = (vaddr & HPAGE_PUD_MASK) + HPAGE_PUD_SIZE;
 
+	pud = pud_mkhuge(pfn_pud(pfn, prot));
 	set_pud_at(mm, vaddr, pudp, pud);
 	pudp_set_wrprotect(mm, vaddr, pudp);
 	pud = READ_ONCE(*pudp);
 	WARN_ON(pud_write(pud));
 
 #ifndef __PAGETABLE_PMD_FOLDED
-	pud = pfn_pud(pfn, prot);
+
+	pud = pud_mkhuge(pfn_pud(pfn, prot));
 	set_pud_at(mm, vaddr, pudp, pud);
 	pudp_huge_get_and_clear(mm, vaddr, pudp);
 	pud = READ_ONCE(*pudp);
 	WARN_ON(!pud_none(pud));
 
-	pud = pfn_pud(pfn, prot);
+	pud = pud_mkhuge(pfn_pud(pfn, prot));
 	set_pud_at(mm, vaddr, pudp, pud);
 	pudp_huge_get_and_clear_full(mm, vaddr, pudp, 1);
 	pud = READ_ONCE(*pudp);
 	WARN_ON(!pud_none(pud));
 #endif /* __PAGETABLE_PMD_FOLDED */
-	pud = pfn_pud(pfn, prot);
+
+	pud = pud_mkhuge(pfn_pud(pfn, prot));
 	pud = pud_wrprotect(pud);
 	pud = pud_mkclean(pud);
 	set_pud_at(mm, vaddr, pudp, pud);
Signature · 1 line
2.26.2
#10 lore raw

PATCH v2 7/13 mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an existing pte entry

In reply to Aneesh Kumar K.V

set_pte_at() should not be used to set a pte entry at locations that already holds a valid pte entry. Architectures like ppc64 don't do TLB invalidate in set_pte_at() and hence expect it to be used to set locations that are not a valid PTE.

---
 mm/debug_vm_pgtable.c | 35 +++++++++++++++--------------------
 1 file changed, 15 insertions(+), 20 deletions(-)
mm/debug_vm_pgtable.c +15 -20
index 76f4c713e5a3..9c7e2c9cfc76 100644
@@ -74,15 +74,18 @@ static void __init pte_advanced_tests(struct mm_struct *mm,
 {
 	pte_t pte = pfn_pte(pfn, prot);
 
+	/*
+	 * Architectures optimize set_pte_at by avoiding TLB flush.
+	 * This requires set_pte_at to be not used to update an
+	 * existing pte entry. Clear pte before we do set_pte_at
+	 */
+
 	pr_debug("Validating PTE advanced\n");
 	pte = pfn_pte(pfn, prot);
 	set_pte_at(mm, vaddr, ptep, pte);
 	ptep_set_wrprotect(mm, vaddr, ptep);
 	pte = ptep_get(ptep);
 	WARN_ON(pte_write(pte));
-
-	pte = pfn_pte(pfn, prot);
-	set_pte_at(mm, vaddr, ptep, pte);
 	ptep_get_and_clear(mm, vaddr, ptep);
 	pte = ptep_get(ptep);
 	WARN_ON(!pte_none(pte));
@@ -96,13 +99,11 @@ static void __init pte_advanced_tests(struct mm_struct *mm,
 	ptep_set_access_flags(vma, vaddr, ptep, pte, 1);
 	pte = ptep_get(ptep);
 	WARN_ON(!(pte_write(pte) && pte_dirty(pte)));
-
-	pte = pfn_pte(pfn, prot);
-	set_pte_at(mm, vaddr, ptep, pte);
 	ptep_get_and_clear_full(mm, vaddr, ptep, 1);
 	pte = ptep_get(ptep);
 	WARN_ON(!pte_none(pte));
 
+	pte = pfn_pte(pfn, prot);
 	pte = pte_mkyoung(pte);
 	set_pte_at(mm, vaddr, ptep, pte);
 	ptep_test_and_clear_young(vma, vaddr, ptep);
@@ -164,9 +165,6 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
 	pmdp_set_wrprotect(mm, vaddr, pmdp);
 	pmd = READ_ONCE(*pmdp);
 	WARN_ON(pmd_write(pmd));
-
-	pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
-	set_pmd_at(mm, vaddr, pmdp, pmd);
 	pmdp_huge_get_and_clear(mm, vaddr, pmdp);
 	pmd = READ_ONCE(*pmdp);
 	WARN_ON(!pmd_none(pmd));
@@ -180,13 +178,11 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
 	pmdp_set_access_flags(vma, vaddr, pmdp, pmd, 1);
 	pmd = READ_ONCE(*pmdp);
 	WARN_ON(!(pmd_write(pmd) && pmd_dirty(pmd)));
-
-	pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
-	set_pmd_at(mm, vaddr, pmdp, pmd);
 	pmdp_huge_get_and_clear_full(vma, vaddr, pmdp, 1);
 	pmd = READ_ONCE(*pmdp);
 	WARN_ON(!pmd_none(pmd));
 
+	pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
 	pmd = pmd_mkyoung(pmd);
 	set_pmd_at(mm, vaddr, pmdp, pmd);
 	pmdp_test_and_clear_young(vma, vaddr, pmdp);
@@ -283,18 +279,10 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
 	WARN_ON(pud_write(pud));
 
 #ifndef __PAGETABLE_PMD_FOLDED
-
-	pud = pud_mkhuge(pfn_pud(pfn, prot));
-	set_pud_at(mm, vaddr, pudp, pud);
 	pudp_huge_get_and_clear(mm, vaddr, pudp);
 	pud = READ_ONCE(*pudp);
 	WARN_ON(!pud_none(pud));
 
-	pud = pud_mkhuge(pfn_pud(pfn, prot));
-	set_pud_at(mm, vaddr, pudp, pud);
-	pudp_huge_get_and_clear_full(mm, vaddr, pudp, 1);
-	pud = READ_ONCE(*pudp);
-	WARN_ON(!pud_none(pud));
 #endif /* __PAGETABLE_PMD_FOLDED */
 
 	pud = pud_mkhuge(pfn_pud(pfn, prot));
@@ -307,6 +295,13 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
 	pud = READ_ONCE(*pudp);
 	WARN_ON(!(pud_write(pud) && pud_dirty(pud)));
 
+#ifndef __PAGETABLE_PMD_FOLDED
+	pudp_huge_get_and_clear_full(vma, vaddr, pudp, 1);
+	pud = READ_ONCE(*pudp);
+	WARN_ON(!pud_none(pud));
+#endif /* __PAGETABLE_PMD_FOLDED */
+
+	pud = pud_mkhuge(pfn_pud(pfn, prot));
 	pud = pud_mkyoung(pud);
 	set_pud_at(mm, vaddr, pudp, pud);
 	pudp_test_and_clear_young(vma, vaddr, pudp);
Signature · 1 line
2.26.2
#11 lore raw

Re: [PATCH v2 07/13] mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an existing pte entry

Le 19/08/2020 à 15:01, Aneesh Kumar K.V a écrit :

Quoted 77 lines

set_pte_at() should not be used to set a pte entry at locations that already holds a valid pte entry. Architectures like ppc64 don't do TLB invalidate in set_pte_at() and hence expect it to be used to set locations that are not a valid PTE.

---
  mm/debug_vm_pgtable.c | 35 +++++++++++++++--------------------
  1 file changed, 15 insertions(+), 20 deletions(-)
mm/debug_vm_pgtable.c +8 -12
index 76f4c713e5a3..9c7e2c9cfc76 100644
@@ -74,15 +74,18 @@ static void __init pte_advanced_tests(struct mm_struct *mm,
  {
  	pte_t pte = pfn_pte(pfn, prot);
 
+	/*
+	 * Architectures optimize set_pte_at by avoiding TLB flush.
+	 * This requires set_pte_at to be not used to update an
+	 * existing pte entry. Clear pte before we do set_pte_at
+	 */
+
  	pr_debug("Validating PTE advanced\n");
  	pte = pfn_pte(pfn, prot);
  	set_pte_at(mm, vaddr, ptep, pte);
  	ptep_set_wrprotect(mm, vaddr, ptep);
  	pte = ptep_get(ptep);
  	WARN_ON(pte_write(pte));
-
-	pte = pfn_pte(pfn, prot);
-	set_pte_at(mm, vaddr, ptep, pte);
  	ptep_get_and_clear(mm, vaddr, ptep);
  	pte = ptep_get(ptep);
  	WARN_ON(!pte_none(pte));
@@ -96,13 +99,11 @@ static void __init pte_advanced_tests(struct mm_struct *mm,
  	ptep_set_access_flags(vma, vaddr, ptep, pte, 1);
  	pte = ptep_get(ptep);
  	WARN_ON(!(pte_write(pte) && pte_dirty(pte)));
-
-	pte = pfn_pte(pfn, prot);
-	set_pte_at(mm, vaddr, ptep, pte);
  	ptep_get_and_clear_full(mm, vaddr, ptep, 1);
  	pte = ptep_get(ptep);
  	WARN_ON(!pte_none(pte));
 
+	pte = pfn_pte(pfn, prot);
  	pte = pte_mkyoung(pte);
  	set_pte_at(mm, vaddr, ptep, pte);
  	ptep_test_and_clear_young(vma, vaddr, ptep);
@@ -164,9 +165,6 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
  	pmdp_set_wrprotect(mm, vaddr, pmdp);
  	pmd = READ_ONCE(*pmdp);
  	WARN_ON(pmd_write(pmd));
-
-	pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
-	set_pmd_at(mm, vaddr, pmdp, pmd);
  	pmdp_huge_get_and_clear(mm, vaddr, pmdp);
  	pmd = READ_ONCE(*pmdp);
  	WARN_ON(!pmd_none(pmd));
@@ -180,13 +178,11 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
  	pmdp_set_access_flags(vma, vaddr, pmdp, pmd, 1);
  	pmd = READ_ONCE(*pmdp);
  	WARN_ON(!(pmd_write(pmd) && pmd_dirty(pmd)));
-
-	pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
-	set_pmd_at(mm, vaddr, pmdp, pmd);
  	pmdp_huge_get_and_clear_full(vma, vaddr, pmdp, 1);
  	pmd = READ_ONCE(*pmdp);
  	WARN_ON(!pmd_none(pmd));
 
+	pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
  	pmd = pmd_mkyoung(pmd);
  	set_pmd_at(mm, vaddr, pmdp, pmd);
  	pmdp_test_and_clear_young(vma, vaddr, pmdp);

mm/debug_vm_pgtable.c static void __init pud_advanced_tests(struct mm_struct *mm, · line 281

Quoted 3 lines
mm/debug_vm_pgtable.c +0 -0
@@ -283,18 +279,10 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
  	WARN_ON(pud_write(pud));
 
  #ifndef __PAGETABLE_PMD_FOLDED

Same as below, once set_put_at() is gone, I don't think this #ifndef __PAGETABLE_PMD_FOLDED is still need, should be possible to replace by 'if (mm_pmd_folded())'

Quoted 16 lines
mm/debug_vm_pgtable.c +0 -8
-
-	pud = pud_mkhuge(pfn_pud(pfn, prot));
-	set_pud_at(mm, vaddr, pudp, pud);
  	pudp_huge_get_and_clear(mm, vaddr, pudp);
  	pud = READ_ONCE(*pudp);
  	WARN_ON(!pud_none(pud));
 
-	pud = pud_mkhuge(pfn_pud(pfn, prot));
-	set_pud_at(mm, vaddr, pudp, pud);
-	pudp_huge_get_and_clear_full(mm, vaddr, pudp, 1);
-	pud = READ_ONCE(*pudp);
-	WARN_ON(!pud_none(pud));
  #endif /* __PAGETABLE_PMD_FOLDED */
 
  	pud = pud_mkhuge(pfn_pud(pfn, prot));

mm/debug_vm_pgtable.c static void __init pud_advanced_tests(struct mm_struct *mm, · line 302

Quoted 8 lines
mm/debug_vm_pgtable.c +5 -0
@@ -307,6 +295,13 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
  	pud = READ_ONCE(*pudp);
  	WARN_ON(!(pud_write(pud) && pud_dirty(pud)));
 
+#ifndef __PAGETABLE_PMD_FOLDED
+	pudp_huge_get_and_clear_full(vma, vaddr, pudp, 1);
+	pud = READ_ONCE(*pudp);
+	WARN_ON(!pud_none(pud));
+#endif /* __PAGETABLE_PMD_FOLDED */

pudp_huge_get_and_clear_full() and pud_none() are always defined, I think this #ifndef can be replaced by an 'if (mm_pmd_folded())'

mm/debug_vm_pgtable.c static void __init pud_advanced_tests(struct mm_struct *mm, · line 307

Quoted 6 lines
mm/debug_vm_pgtable.c +2 -0
+
+	pud = pud_mkhuge(pfn_pud(pfn, prot));
  	pud = pud_mkyoung(pud);
  	set_pud_at(mm, vaddr, pudp, pud);
  	pudp_test_and_clear_young(vma, vaddr, pudp);

Christophe

#12 lore raw

Re: [PATCH v2 07/13] mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an existing pte entry

Quoted 78 lines · Christophe Leroy

Le 19/08/2020 à 15:01, Aneesh Kumar K.V a écrit :

Quoted 75 lines

set_pte_at() should not be used to set a pte entry at locations that already holds a valid pte entry. Architectures like ppc64 don't do TLB invalidate in set_pte_at() and hence expect it to be used to set locations that are not a valid PTE.

---
  mm/debug_vm_pgtable.c | 35 +++++++++++++++--------------------

  1 file changed, 15 insertions(+), 20 deletions(-)
mm/debug_vm_pgtable.c +7 -9
index 76f4c713e5a3..9c7e2c9cfc76 100644
@@ -74,15 +74,18 @@ static void __init pte_advanced_tests(struct 
 mm_struct *mm,
   {
       pte_t pte = pfn_pte(pfn, prot);
+    /*
+     * Architectures optimize set_pte_at by avoiding TLB flush.
+     * This requires set_pte_at to be not used to update an
+     * existing pte entry. Clear pte before we do set_pte_at
+     */
+
       pr_debug("Validating PTE advanced\n");
       pte = pfn_pte(pfn, prot);
       set_pte_at(mm, vaddr, ptep, pte);
       ptep_set_wrprotect(mm, vaddr, ptep);
       pte = ptep_get(ptep);
       WARN_ON(pte_write(pte));
-
-    pte = pfn_pte(pfn, prot);
-    set_pte_at(mm, vaddr, ptep, pte);
       ptep_get_and_clear(mm, vaddr, ptep);
       pte = ptep_get(ptep);
       WARN_ON(!pte_none(pte));
@@ -96,13 +99,11 @@ static void __init pte_advanced_tests(struct 
 mm_struct *mm,
       ptep_set_access_flags(vma, vaddr, ptep, pte, 1);
       pte = ptep_get(ptep);
       WARN_ON(!(pte_write(pte) && pte_dirty(pte)));
-
-    pte = pfn_pte(pfn, prot);
-    set_pte_at(mm, vaddr, ptep, pte);
       ptep_get_and_clear_full(mm, vaddr, ptep, 1);
       pte = ptep_get(ptep);
       WARN_ON(!pte_none(pte));
+    pte = pfn_pte(pfn, prot);
       pte = pte_mkyoung(pte);
       set_pte_at(mm, vaddr, ptep, pte);
       ptep_test_and_clear_young(vma, vaddr, ptep);
@@ -164,9 +165,6 @@ static void __init pmd_advanced_tests(struct 
 mm_struct *mm,
       pmdp_set_wrprotect(mm, vaddr, pmdp);
       pmd = READ_ONCE(*pmdp);
       WARN_ON(pmd_write(pmd));
-
-    pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
-    set_pmd_at(mm, vaddr, pmdp, pmd);
       pmdp_huge_get_and_clear(mm, vaddr, pmdp);
       pmd = READ_ONCE(*pmdp);

      WARN_ON(!pmd_none(pmd));

mm/debug_vm_pgtable.c +1 -3
@@ -180,13 +178,11 @@ static void __init pmd_advanced_tests(struct 
 mm_struct *mm,
       pmdp_set_access_flags(vma, vaddr, pmdp, pmd, 1);
       pmd = READ_ONCE(*pmdp);
       WARN_ON(!(pmd_write(pmd) && pmd_dirty(pmd)));
-
-    pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
-    set_pmd_at(mm, vaddr, pmdp, pmd);
       pmdp_huge_get_and_clear_full(vma, vaddr, pmdp, 1);
       pmd = READ_ONCE(*pmdp);
       WARN_ON(!pmd_none(pmd));
+    pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
       pmd = pmd_mkyoung(pmd);
Quoted 8 lines

mm/debug_vm_pgtable.c static void __init pud_advanced_tests(struct · line 281

Quoted 6 lines
mm/debug_vm_pgtable.c +0 -0
       set_pmd_at(mm, vaddr, pmdp, pmd);
       pmdp_test_and_clear_young(vma, vaddr, pmdp);
@@ -283,18 +279,10 @@ static void __init pud_advanced_tests(struct 
 mm_struct *mm,
       WARN_ON(pud_write(pud));
   #ifndef __PAGETABLE_PMD_FOLDED

Same as below, once set_put_at() is gone, I don't think this #ifndef __PAGETABLE_PMD_FOLDED is still need, should be possible to replace by 'if (mm_pmd_folded())'

I would skip that change in this series because I still haven't worked out what it means to have FOLDED PMD with CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD.

We should probably push that as a cleanup later and somebody who can test that config can do that? Currently i can't boot ppc64 with DBUG_VM_PGTABLE enabled on ppc64 because it is all buggy w.r.t rules.

-aneesh

#13 lore raw

Re: [PATCH v2 07/13] mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an existing pte entry

Quoted 83 lines · Aneesh Kumar K.V
Quoted 73 lines · Christophe Leroy

Le 19/08/2020 à 15:01, Aneesh Kumar K.V a écrit :

Quoted 70 lines

set_pte_at() should not be used to set a pte entry at locations that already holds a valid pte entry. Architectures like ppc64 don't do TLB invalidate in set_pte_at() and hence expect it to be used to set locations that are not a valid PTE.

---
  mm/debug_vm_pgtable.c | 35 +++++++++++++++--------------------

  1 file changed, 15 insertions(+), 20 deletions(-)
mm/debug_vm_pgtable.c +8 -12
index 76f4c713e5a3..9c7e2c9cfc76 100644
@@ -74,15 +74,18 @@ static void __init pte_advanced_tests(struct mm_struct *mm,
   {
       pte_t pte = pfn_pte(pfn, prot);
+    /*
+     * Architectures optimize set_pte_at by avoiding TLB flush.
+     * This requires set_pte_at to be not used to update an
+     * existing pte entry. Clear pte before we do set_pte_at
+     */
+
       pr_debug("Validating PTE advanced\n");
       pte = pfn_pte(pfn, prot);
       set_pte_at(mm, vaddr, ptep, pte);
       ptep_set_wrprotect(mm, vaddr, ptep);
       pte = ptep_get(ptep);
       WARN_ON(pte_write(pte));
-
-    pte = pfn_pte(pfn, prot);
-    set_pte_at(mm, vaddr, ptep, pte);
       ptep_get_and_clear(mm, vaddr, ptep);
       pte = ptep_get(ptep);
       WARN_ON(!pte_none(pte));
@@ -96,13 +99,11 @@ static void __init pte_advanced_tests(struct mm_struct *mm,
       ptep_set_access_flags(vma, vaddr, ptep, pte, 1);
       pte = ptep_get(ptep);
       WARN_ON(!(pte_write(pte) && pte_dirty(pte)));
-
-    pte = pfn_pte(pfn, prot);
-    set_pte_at(mm, vaddr, ptep, pte);
       ptep_get_and_clear_full(mm, vaddr, ptep, 1);
       pte = ptep_get(ptep);
       WARN_ON(!pte_none(pte));
+    pte = pfn_pte(pfn, prot);
       pte = pte_mkyoung(pte);
       set_pte_at(mm, vaddr, ptep, pte);
       ptep_test_and_clear_young(vma, vaddr, ptep);
@@ -164,9 +165,6 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
       pmdp_set_wrprotect(mm, vaddr, pmdp);
       pmd = READ_ONCE(*pmdp);
       WARN_ON(pmd_write(pmd));
-
-    pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
-    set_pmd_at(mm, vaddr, pmdp, pmd);
       pmdp_huge_get_and_clear(mm, vaddr, pmdp);
       pmd = READ_ONCE(*pmdp);
       WARN_ON(!pmd_none(pmd));
@@ -180,13 +178,11 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
       pmdp_set_access_flags(vma, vaddr, pmdp, pmd, 1);
       pmd = READ_ONCE(*pmdp);
       WARN_ON(!(pmd_write(pmd) && pmd_dirty(pmd)));
-
-    pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
-    set_pmd_at(mm, vaddr, pmdp, pmd);
       pmdp_huge_get_and_clear_full(vma, vaddr, pmdp, 1);
       pmd = READ_ONCE(*pmdp);
       WARN_ON(!pmd_none(pmd));
+    pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
Quoted 8 lines

mm/debug_vm_pgtable.c static void __init pud_advanced_tests(struct mm_struct *mm, · line 280

Quoted 6 lines
mm/debug_vm_pgtable.c +0 -0
       pmd = pmd_mkyoung(pmd);
       set_pmd_at(mm, vaddr, pmdp, pmd);
       pmdp_test_and_clear_young(vma, vaddr, pmdp);
@@ -283,18 +279,10 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
       WARN_ON(pud_write(pud));
   #ifndef __PAGETABLE_PMD_FOLDED

Same as below, once set_put_at() is gone, I don't think this #ifndef __PAGETABLE_PMD_FOLDED is still need, should be possible to replace by 'if (mm_pmd_folded())'

Quoted 4 lines

I would skip that change in this series because I still haven't worked out what it means to have FOLDED PMD with CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD.

We should probably push that as a cleanup later and somebody who can test that config can do that? Currently i can't boot ppc64 with DBUG_VM_PGTABLE enabled on ppc64 because it is all buggy w.r.t rules.

Agreed. I think its OK not address these changes/improvements in this particular series which is trying to modify the test to make it run on ppc64 platform. I will probably look into that later.

#14 lore raw

PATCH v2 8/13 mm/debug_vm_pgtable/thp: Use page table depost/withdraw with THP

In reply to Aneesh Kumar K.V

Architectures like ppc64 use deposited page table while updating the huge pte entries.

---
 mm/debug_vm_pgtable.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
mm/debug_vm_pgtable.c +6 -2
index 9c7e2c9cfc76..6dcac2b40fef 100644
@@ -149,7 +149,7 @@ static void __init pmd_basic_tests(unsigned long pfn, pgprot_t prot)
 static void __init pmd_advanced_tests(struct mm_struct *mm,
 				      struct vm_area_struct *vma, pmd_t *pmdp,
 				      unsigned long pfn, unsigned long vaddr,
-				      pgprot_t prot)
+				      pgprot_t prot, pgtable_t pgtable)
 {
 	pmd_t pmd;
 
@@ -160,6 +160,8 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
 	/* Align the address wrt HPAGE_PMD_SIZE */
 	vaddr = (vaddr & HPAGE_PMD_MASK) + HPAGE_PMD_SIZE;
 
+	pgtable_trans_huge_deposit(mm, pmdp, pgtable);
+
 	pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
 	set_pmd_at(mm, vaddr, pmdp, pmd);
 	pmdp_set_wrprotect(mm, vaddr, pmdp);
@@ -188,6 +190,8 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
 	pmdp_test_and_clear_young(vma, vaddr, pmdp);
 	pmd = READ_ONCE(*pmdp);
 	WARN_ON(pmd_young(pmd));
+
+	pgtable = pgtable_trans_huge_withdraw(mm, pmdp);
 }
 
 static void __init pmd_leaf_tests(unsigned long pfn, pgprot_t prot)
@@ -1000,7 +1004,7 @@ static int __init debug_vm_pgtable(void)
 	pgd_clear_tests(mm, pgdp);
 
 	pte_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
-	pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot);
+	pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
 	pud_advanced_tests(mm, vma, pudp, pud_aligned, vaddr, prot);
 	hugetlb_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
 
Signature · 1 line
2.26.2
#15 lore raw

PATCH v2 9/13 mm/debug_vm_pgtable/locks: Move non page table modifying test together

In reply to Aneesh Kumar K.V

This will help in adding proper locks in a later patch

---
 mm/debug_vm_pgtable.c | 52 ++++++++++++++++++++++++-------------------
 1 file changed, 29 insertions(+), 23 deletions(-)
mm/debug_vm_pgtable.c +29 -23
index 6dcac2b40fef..69fe3cd8126c 100644
@@ -977,7 +977,7 @@ static int __init debug_vm_pgtable(void)
 	p4dp = p4d_alloc(mm, pgdp, vaddr);
 	pudp = pud_alloc(mm, p4dp, vaddr);
 	pmdp = pmd_alloc(mm, pudp, vaddr);
-	ptep = pte_alloc_map_lock(mm, pmdp, vaddr, &ptl);
+	ptep = pte_alloc_map(mm, pmdp, vaddr);
 
 	/*
 	 * Save all the page table page addresses as the page table
@@ -997,33 +997,12 @@ static int __init debug_vm_pgtable(void)
 	p4d_basic_tests(p4d_aligned, prot);
 	pgd_basic_tests(pgd_aligned, prot);
 
-	pte_clear_tests(mm, ptep, vaddr);
-	pmd_clear_tests(mm, pmdp);
-	pud_clear_tests(mm, pudp);
-	p4d_clear_tests(mm, p4dp);
-	pgd_clear_tests(mm, pgdp);
-
-	pte_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
-	pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
-	pud_advanced_tests(mm, vma, pudp, pud_aligned, vaddr, prot);
-	hugetlb_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
-
 	pmd_leaf_tests(pmd_aligned, prot);
 	pud_leaf_tests(pud_aligned, prot);
 
-	pmd_huge_tests(pmdp, pmd_aligned, prot);
-	pud_huge_tests(pudp, pud_aligned, prot);
-
 	pte_savedwrite_tests(pte_aligned, protnone);
 	pmd_savedwrite_tests(pmd_aligned, protnone);
 
-	pte_unmap_unlock(ptep, ptl);
-
-	pmd_populate_tests(mm, pmdp, saved_ptep);
-	pud_populate_tests(mm, pudp, saved_pmdp);
-	p4d_populate_tests(mm, p4dp, saved_pudp);
-	pgd_populate_tests(mm, pgdp, saved_p4dp);
-
 	pte_special_tests(pte_aligned, prot);
 	pte_protnone_tests(pte_aligned, protnone);
 	pmd_protnone_tests(pmd_aligned, protnone);
@@ -1041,11 +1020,38 @@ static int __init debug_vm_pgtable(void)
 	pmd_swap_tests(pmd_aligned, prot);
 
 	swap_migration_tests();
-	hugetlb_basic_tests(pte_aligned, prot);
 
 	pmd_thp_tests(pmd_aligned, prot);
 	pud_thp_tests(pud_aligned, prot);
 
+	/*
+	 * Page table modifying tests
+	 */
+	pte_clear_tests(mm, ptep, vaddr);
+	pmd_clear_tests(mm, pmdp);
+	pud_clear_tests(mm, pudp);
+	p4d_clear_tests(mm, p4dp);
+	pgd_clear_tests(mm, pgdp);
+
+	ptep = pte_alloc_map_lock(mm, pmdp, vaddr, &ptl);
+	pte_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
+	pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
+	pud_advanced_tests(mm, vma, pudp, pud_aligned, vaddr, prot);
+	hugetlb_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
+
+
+	pmd_huge_tests(pmdp, pmd_aligned, prot);
+	pud_huge_tests(pudp, pud_aligned, prot);
+
+	pte_unmap_unlock(ptep, ptl);
+
+	pmd_populate_tests(mm, pmdp, saved_ptep);
+	pud_populate_tests(mm, pudp, saved_pmdp);
+	p4d_populate_tests(mm, p4dp, saved_pudp);
+	pgd_populate_tests(mm, pgdp, saved_p4dp);
+
+	hugetlb_basic_tests(pte_aligned, prot);
+
 	p4d_free(mm, saved_p4dp);
 	pud_free(mm, saved_pudp);
 	pmd_free(mm, saved_pmdp);
Signature · 1 line
2.26.2
#16 lore raw

PATCH v2 10/13 mm/debug_vm_pgtable/locks: Take correct page table lock

In reply to Aneesh Kumar K.V

Make sure we call pte accessors with correct lock held.

---
 mm/debug_vm_pgtable.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)
mm/debug_vm_pgtable.c +20 -14
index 69fe3cd8126c..8f7a8ccb5a54 100644
@@ -1024,33 +1024,39 @@ static int __init debug_vm_pgtable(void)
 	pmd_thp_tests(pmd_aligned, prot);
 	pud_thp_tests(pud_aligned, prot);
 
+	hugetlb_basic_tests(pte_aligned, prot);
+
 	/*
 	 * Page table modifying tests
 	 */
-	pte_clear_tests(mm, ptep, vaddr);
-	pmd_clear_tests(mm, pmdp);
-	pud_clear_tests(mm, pudp);
-	p4d_clear_tests(mm, p4dp);
-	pgd_clear_tests(mm, pgdp);
 
 	ptep = pte_alloc_map_lock(mm, pmdp, vaddr, &ptl);
+	pte_clear_tests(mm, ptep, vaddr);
 	pte_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
-	pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
-	pud_advanced_tests(mm, vma, pudp, pud_aligned, vaddr, prot);
-	hugetlb_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
-
+	pte_unmap_unlock(ptep, ptl);
 
+	ptl = pmd_lock(mm, pmdp);
+	pmd_clear_tests(mm, pmdp);
+	pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
 	pmd_huge_tests(pmdp, pmd_aligned, prot);
+	pmd_populate_tests(mm, pmdp, saved_ptep);
+	spin_unlock(ptl);
+
+	ptl = pud_lock(mm, pudp);
+	pud_clear_tests(mm, pudp);
+	pud_advanced_tests(mm, vma, pudp, pud_aligned, vaddr, prot);
 	pud_huge_tests(pudp, pud_aligned, prot);
+	pud_populate_tests(mm, pudp, saved_pmdp);
+	spin_unlock(ptl);
 
-	pte_unmap_unlock(ptep, ptl);
+	//hugetlb_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
 
-	pmd_populate_tests(mm, pmdp, saved_ptep);
-	pud_populate_tests(mm, pudp, saved_pmdp);
+	spin_lock(&mm->page_table_lock);
+	p4d_clear_tests(mm, p4dp);
+	pgd_clear_tests(mm, pgdp);
 	p4d_populate_tests(mm, p4dp, saved_pudp);
 	pgd_populate_tests(mm, pgdp, saved_p4dp);
-
-	hugetlb_basic_tests(pte_aligned, prot);
+	spin_unlock(&mm->page_table_lock);
 
 	p4d_free(mm, saved_p4dp);
 	pud_free(mm, saved_pudp);
Signature · 1 line
2.26.2
#17 lore raw

Re: [PATCH v2 10/13] mm/debug_vm_pgtable/locks: Take correct page table lock

Quoted 41 lines · Aneesh Kumar K.V

Make sure we call pte accessors with correct lock held.

---
 mm/debug_vm_pgtable.c | 34 ++++++++++++++++++++--------------
 1 file changed, 20 insertions(+), 14 deletions(-)
mm/debug_vm_pgtable.c +9 -9
index 69fe3cd8126c..8f7a8ccb5a54 100644
@@ -1024,33 +1024,39 @@ static int __init debug_vm_pgtable(void)
 	pmd_thp_tests(pmd_aligned, prot);
 	pud_thp_tests(pud_aligned, prot);
 
+	hugetlb_basic_tests(pte_aligned, prot);
+
 	/*
 	 * Page table modifying tests
 	 */
-	pte_clear_tests(mm, ptep, vaddr);
-	pmd_clear_tests(mm, pmdp);
-	pud_clear_tests(mm, pudp);
-	p4d_clear_tests(mm, p4dp);
-	pgd_clear_tests(mm, pgdp);
 
 	ptep = pte_alloc_map_lock(mm, pmdp, vaddr, &ptl);
+	pte_clear_tests(mm, ptep, vaddr);
 	pte_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
-	pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
-	pud_advanced_tests(mm, vma, pudp, pud_aligned, vaddr, prot);
-	hugetlb_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
-
+	pte_unmap_unlock(ptep, ptl);
 
+	ptl = pmd_lock(mm, pmdp);
+	pmd_clear_tests(mm, pmdp);
+	pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
 	pmd_huge_tests(pmdp, pmd_aligned, prot);
+	pmd_populate_tests(mm, pmdp, saved_ptep);
+	spin_unlock(ptl);

mm/debug_vm_pgtable.c static int __init debug_vm_pgtable(void) · line 1052

Quoted 10 lines
mm/debug_vm_pgtable.c +7 -1
+
+	ptl = pud_lock(mm, pudp);
+	pud_clear_tests(mm, pudp);
+	pud_advanced_tests(mm, vma, pudp, pud_aligned, vaddr, prot);
 	pud_huge_tests(pudp, pud_aligned, prot);
+	pud_populate_tests(mm, pudp, saved_pmdp);
+	spin_unlock(ptl);
 
-	pte_unmap_unlock(ptep, ptl);
+	//hugetlb_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);

Commenting out an existing test in the middle of another change ?

#18 lore raw

Re: [PATCH v2 10/13] mm/debug_vm_pgtable/locks: Take correct page table lock

Quoted 52 lines · Anshuman Khandual
Quoted 49 lines · Aneesh Kumar K.V

Make sure we call pte accessors with correct lock held.

---
  mm/debug_vm_pgtable.c | 34 ++++++++++++++++++++--------------
  1 file changed, 20 insertions(+), 14 deletions(-)
mm/debug_vm_pgtable.c +15 -9
index 69fe3cd8126c..8f7a8ccb5a54 100644
@@ -1024,33 +1024,39 @@ static int __init debug_vm_pgtable(void)
  	pmd_thp_tests(pmd_aligned, prot);
  	pud_thp_tests(pud_aligned, prot);
 
+	hugetlb_basic_tests(pte_aligned, prot);
+
  	/*
  	 * Page table modifying tests
  	 */
-	pte_clear_tests(mm, ptep, vaddr);
-	pmd_clear_tests(mm, pmdp);
-	pud_clear_tests(mm, pudp);
-	p4d_clear_tests(mm, p4dp);
-	pgd_clear_tests(mm, pgdp);
 
  	ptep = pte_alloc_map_lock(mm, pmdp, vaddr, &ptl);
+	pte_clear_tests(mm, ptep, vaddr);
  	pte_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
-	pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
-	pud_advanced_tests(mm, vma, pudp, pud_aligned, vaddr, prot);
-	hugetlb_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
-
+	pte_unmap_unlock(ptep, ptl);
 
+	ptl = pmd_lock(mm, pmdp);
+	pmd_clear_tests(mm, pmdp);
+	pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
  	pmd_huge_tests(pmdp, pmd_aligned, prot);
+	pmd_populate_tests(mm, pmdp, saved_ptep);
+	spin_unlock(ptl);
+
+	ptl = pud_lock(mm, pudp);
+	pud_clear_tests(mm, pudp);
+	pud_advanced_tests(mm, vma, pudp, pud_aligned, vaddr, prot);
  	pud_huge_tests(pudp, pud_aligned, prot);
+	pud_populate_tests(mm, pudp, saved_pmdp);
+	spin_unlock(ptl);
 
Quoted 5 lines

mm/debug_vm_pgtable.c static int __init debug_vm_pgtable(void) · line 1052

Quoted 2 lines
mm/debug_vm_pgtable.c +1 -1
-	pte_unmap_unlock(ptep, ptl);
+	//hugetlb_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);

Commenting out an existing test in the middle of another change ?

That is already fixed. That was me creating a git diff against a wrong branch.

Thanks.
-aneesh

#19 lore raw

PATCH v2 11/13 mm/debug_vm_pgtable/pmd_clear: Don't use pmd/pud_clear on pte entries

In reply to Aneesh Kumar K.V

pmd_clear() should not be used to clear pmd level pte entries.

---
 mm/debug_vm_pgtable.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
mm/debug_vm_pgtable.c +4 -3
index 8f7a8ccb5a54..63576fe767a2 100644
@@ -191,6 +191,8 @@ static void __init pmd_advanced_tests(struct mm_struct *mm,
 	pmd = READ_ONCE(*pmdp);
 	WARN_ON(pmd_young(pmd));
 
+	/*  Clear the pte entries  */
+	pmdp_huge_get_and_clear(mm, vaddr, pmdp);
 	pgtable = pgtable_trans_huge_withdraw(mm, pmdp);
 }
 
@@ -311,6 +313,8 @@ static void __init pud_advanced_tests(struct mm_struct *mm,
 	pudp_test_and_clear_young(vma, vaddr, pudp);
 	pud = READ_ONCE(*pudp);
 	WARN_ON(pud_young(pud));
+
+	pudp_huge_get_and_clear(mm, vaddr, pudp);
 }
 
 static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot)
@@ -429,8 +433,6 @@ static void __init pud_populate_tests(struct mm_struct *mm, pud_t *pudp,
 	 * This entry points to next level page table page.
 	 * Hence this must not qualify as pud_bad().
 	 */
-	pmd_clear(pmdp);
-	pud_clear(pudp);
 	pud_populate(mm, pudp, pmdp);
 	pud = READ_ONCE(*pudp);
 	WARN_ON(pud_bad(pud));
@@ -562,7 +564,6 @@ static void __init pmd_populate_tests(struct mm_struct *mm, pmd_t *pmdp,
 	 * This entry points to next level page table page.
 	 * Hence this must not qualify as pmd_bad().
 	 */
-	pmd_clear(pmdp);
 	pmd_populate(mm, pmdp, pgtable);
 	pmd = READ_ONCE(*pmdp);
 	WARN_ON(pmd_bad(pmd));
Signature · 1 line
2.26.2
#20 lore raw

PATCH v2 12/13 mm/debug_vm_pgtable/hugetlb: Disable hugetlb test on ppc64

In reply to Aneesh Kumar K.V

The seems to be missing quite a lot of details w.r.t allocating the correct pgtable_t page (huge_pte_alloc()), holding the right lock (huge_pte_lock()) etc. The vma used is also not a hugetlb VMA.

ppc64 do have runtime checks within CONFIG_DEBUG_VM for most of these. Hence disable the test on ppc64.

---
 mm/debug_vm_pgtable.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
mm/debug_vm_pgtable.c +5 -1
index 63576fe767a2..09ce9974c187 100644
@@ -798,6 +798,7 @@ static void __init hugetlb_basic_tests(unsigned long pfn, pgprot_t prot)
 #endif /* CONFIG_ARCH_WANT_GENERAL_HUGETLB */
 }
 
+#ifndef CONFIG_PPC_BOOK3S_64
 static void __init hugetlb_advanced_tests(struct mm_struct *mm,
 					  struct vm_area_struct *vma,
 					  pte_t *ptep, unsigned long pfn,
@@ -840,6 +841,7 @@ static void __init hugetlb_advanced_tests(struct mm_struct *mm,
 	pte = huge_ptep_get(ptep);
 	WARN_ON(!(huge_pte_write(pte) && huge_pte_dirty(pte)));
 }
+#endif
 #else  /* !CONFIG_HUGETLB_PAGE */
 static void __init hugetlb_basic_tests(unsigned long pfn, pgprot_t prot) { }
 static void __init hugetlb_advanced_tests(struct mm_struct *mm,
@@ -1050,7 +1052,9 @@ static int __init debug_vm_pgtable(void)
 	pud_populate_tests(mm, pudp, saved_pmdp);
 	spin_unlock(ptl);
 
-	//hugetlb_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
+#ifndef CONFIG_PPC_BOOK3S_64
+	hugetlb_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
+#endif
 
 	spin_lock(&mm->page_table_lock);
 	p4d_clear_tests(mm, p4dp);
Signature · 1 line
2.26.2
#21 lore raw

PATCH v2 13/13 mm/debug_vm_pgtable: populate a pte entry before fetching it

In reply to Aneesh Kumar K.V

pte_clear_tests operate on an existing pte entry. Make sure that is not a none pte entry.

---
 mm/debug_vm_pgtable.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
mm/debug_vm_pgtable.c +4 -2
index 09ce9974c187..7d9f8e1d790f 100644
@@ -531,7 +531,7 @@ static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
 static void __init pte_clear_tests(struct mm_struct *mm, pte_t *ptep,
 				   unsigned long vaddr)
 {
-	pte_t pte = ptep_get(ptep);
+	pte_t pte =  ptep_get_and_clear(mm, vaddr, ptep);
 
 	pr_debug("Validating PTE clear\n");
 	pte = __pte(pte_val(pte) | RANDOM_ORVALUE);
@@ -929,7 +929,7 @@ static int __init debug_vm_pgtable(void)
 	p4d_t *p4dp, *saved_p4dp;
 	pud_t *pudp, *saved_pudp;
 	pmd_t *pmdp, *saved_pmdp, pmd;
-	pte_t *ptep;
+	pte_t *ptep, pte;
 	pgtable_t saved_ptep;
 	pgprot_t prot, protnone;
 	phys_addr_t paddr;
@@ -1034,6 +1034,8 @@ static int __init debug_vm_pgtable(void)
 	 */
 
 	ptep = pte_alloc_map_lock(mm, pmdp, vaddr, &ptl);
+	pte = pfn_pte(pte_aligned, prot);
+	set_pte_at(mm, vaddr, ptep, pte);
 	pte_clear_tests(mm, ptep, vaddr);
 	pte_advanced_tests(mm, vma, ptep, pte_aligned, vaddr, prot);
 	pte_unmap_unlock(ptep, ptl);
Signature · 1 line
2.26.2
#22 lore raw
In reply to Aneesh Kumar K.V
Quoted 32 lines · Aneesh Kumar K.V

This patch series includes fixes for debug_vm_pgtable test code so that they follow page table updates rules correctly. The first two patches introduce changes w.r.t ppc64. The patches are included in this series for completeness. We can merge them via ppc64 tree if required.

Hugetlb test is disabled on ppc64 because that needs larger change to satisfy page table update rules.

Changes from V1:
* Address review feedback
* drop test specific pfn_pte and pfn_pmd.
* Update ppc64 page table helper to add _PAGE_PTE

Aneesh Kumar K.V (13):
powerpc/mm: Add DEBUG_VM WARN for pmd_clear
powerpc/mm: Move setting pte specific flags to pfn_pte mm/debug_vm_pgtable/ppc64: Avoid setting top bits in radom value mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge
vmap support.
mm/debug_vm_pgtable/savedwrite: Enable savedwrite test with
CONFIG_NUMA_BALANCING
mm/debug_vm_pgtable/THP: Mark the pte entry huge before using
set_pmd/pud_at
mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an
existing pte entry
mm/debug_vm_pgtable/thp: Use page table depost/withdraw with THP mm/debug_vm_pgtable/locks: Move non page table modifying test together mm/debug_vm_pgtable/locks: Take correct page table lock mm/debug_vm_pgtable/pmd_clear: Don't use pmd/pud_clear on pte entries mm/debug_vm_pgtable/hugetlb: Disable hugetlb test on ppc64 mm/debug_vm_pgtable: populate a pte entry before fetching it

Quoted 8 lines
 arch/powerpc/include/asm/book3s/64/pgtable.h |  29 +++-
 arch/powerpc/include/asm/nohash/pgtable.h    |   5 -
 arch/powerpc/mm/book3s64/pgtable.c           |   2 +-
 arch/powerpc/mm/pgtable.c                    |   5 -
 include/linux/io.h                           |  12 ++
 mm/debug_vm_pgtable.c                        | 151 +++++++++++--------
 6 files changed, 127 insertions(+), 77 deletions(-)

BTW I picked a wrong branch when sending this. Attaching the diff against what I want to send. pfn_pmd() no more updates _PAGE_PTE because that is handled by pmd_mkhuge().

arch/powerpc/mm/book3s64/pgtable.c +1 -1
index 3b4da7c63e28..e18ae50a275c 100644
@@ -141,7 +141,7 @@ pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot)
 	unsigned long pmdv;
 
 	pmdv = (pfn << PAGE_SHIFT) & PTE_RPN_MASK;
-	return __pmd(pmdv | pgprot_val(pgprot) | _PAGE_PTE);
+	return pmd_set_protbits(__pmd(pmdv), pgprot);
 }
 
 pmd_t mk_pmd(struct page *page, pgprot_t pgprot)
mm/debug_vm_pgtable.c +1 -1
index 7d9f8e1d790f..cad61d22f33a 100644
@@ -229,7 +229,7 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
 
 static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
 {
-	pmd_t pmd = pfn_pmd(pfn, prot);
+	pmd_t pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
 
 	if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
 		return;
#23 lore raw
Quoted 66 lines · Aneesh Kumar K.V
Quoted 32 lines · Aneesh Kumar K.V

This patch series includes fixes for debug_vm_pgtable test code so that they follow page table updates rules correctly. The first two patches introduce changes w.r.t ppc64. The patches are included in this series for completeness. We can merge them via ppc64 tree if required.

Hugetlb test is disabled on ppc64 because that needs larger change to satisfy page table update rules.

Changes from V1:
* Address review feedback
* drop test specific pfn_pte and pfn_pmd.
* Update ppc64 page table helper to add _PAGE_PTE

Aneesh Kumar K.V (13):
powerpc/mm: Add DEBUG_VM WARN for pmd_clear
powerpc/mm: Move setting pte specific flags to pfn_pte mm/debug_vm_pgtable/ppc64: Avoid setting top bits in radom value mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge
vmap support.
mm/debug_vm_pgtable/savedwrite: Enable savedwrite test with
CONFIG_NUMA_BALANCING
mm/debug_vm_pgtable/THP: Mark the pte entry huge before using
set_pmd/pud_at
mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an
existing pte entry
mm/debug_vm_pgtable/thp: Use page table depost/withdraw with THP mm/debug_vm_pgtable/locks: Move non page table modifying test together mm/debug_vm_pgtable/locks: Take correct page table lock mm/debug_vm_pgtable/pmd_clear: Don't use pmd/pud_clear on pte entries mm/debug_vm_pgtable/hugetlb: Disable hugetlb test on ppc64 mm/debug_vm_pgtable: populate a pte entry before fetching it

Quoted 8 lines
 arch/powerpc/include/asm/book3s/64/pgtable.h |  29 +++-
 arch/powerpc/include/asm/nohash/pgtable.h    |   5 -
 arch/powerpc/mm/book3s64/pgtable.c           |   2 +-
 arch/powerpc/mm/pgtable.c                    |   5 -
 include/linux/io.h                           |  12 ++
 mm/debug_vm_pgtable.c                        | 151 +++++++++++--------
 6 files changed, 127 insertions(+), 77 deletions(-)

BTW I picked a wrong branch when sending this. Attaching the diff against what I want to send. pfn_pmd() no more updates _PAGE_PTE because that is handled by pmd_mkhuge().

arch/powerpc/mm/book3s64/pgtable.c +1 -1
index 3b4da7c63e28..e18ae50a275c 100644
@@ -141,7 +141,7 @@ pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot)
 	unsigned long pmdv;
 
 	pmdv = (pfn << PAGE_SHIFT) & PTE_RPN_MASK;
-	return __pmd(pmdv | pgprot_val(pgprot) | _PAGE_PTE);
+	return pmd_set_protbits(__pmd(pmdv), pgprot);
 }
 
 pmd_t mk_pmd(struct page *page, pgprot_t pgprot)
mm/debug_vm_pgtable.c +0 -0
index 7d9f8e1d790f..cad61d22f33a 100644

mm/debug_vm_pgtable.c static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) · line 235

Quoted 8 lines
mm/debug_vm_pgtable.c +1 -1
@@ -229,7 +229,7 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
 
 static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
 {
-	pmd_t pmd = pfn_pmd(pfn, prot);
+	pmd_t pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
 
 	if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
 		return;

Cover letter does not mention which branch or tag this series applies on. Just assumed it to be 5.9-rc1. Should the above changes be captured as a pre-requisite patch ?

Anyways, the series fails to be build on arm64.

A) Without CONFIG_TRANSPARENT_HUGEPAGE

mm/debug_vm_pgtable.c: In function ‘debug_vm_pgtable’:
mm/debug_vm_pgtable.c:1045:2: error: too many arguments to function ‘pmd_advanced_tests’
pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep); ^~~~~~~~~~~~~~~~~~
mm/debug_vm_pgtable.c:366:20: note: declared here
static void __init pmd_advanced_tests(struct mm_struct *mm,

                    ^~~~~~~~~~~~~~~~~~

B) As mentioned previously, this should be solved by including <linux/io.h>

mm/debug_vm_pgtable.c: In function ‘pmd_huge_tests’:
mm/debug_vm_pgtable.c:215:7: error: implicit declaration of function ‘arch_ioremap_pmd_supported’; did you mean ‘arch_disable_smp_support’? [-Werror=implicit-function-declaration]
if (!arch_ioremap_pmd_supported())

       ^~~~~~~~~~~~~~~~~~~~~~~~~~

Please make sure that the series builds on all enabled platforms i.e x86, arm64, ppc32, ppc64, arc, s390 along with selectively enabling/disabling all the features that make various #ifdefs in the test.

- Anshuman

#24 lore raw
Quoted 99 lines · Anshuman Khandual
Quoted 66 lines · Aneesh Kumar K.V
Quoted 32 lines · Aneesh Kumar K.V

This patch series includes fixes for debug_vm_pgtable test code so that they follow page table updates rules correctly. The first two patches introduce changes w.r.t ppc64. The patches are included in this series for completeness. We can merge them via ppc64 tree if required.

Hugetlb test is disabled on ppc64 because that needs larger change to satisfy page table update rules.

Changes from V1:
* Address review feedback
* drop test specific pfn_pte and pfn_pmd.
* Update ppc64 page table helper to add _PAGE_PTE

Aneesh Kumar K.V (13):
powerpc/mm: Add DEBUG_VM WARN for pmd_clear
powerpc/mm: Move setting pte specific flags to pfn_pte mm/debug_vm_pgtable/ppc64: Avoid setting top bits in radom value mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge
vmap support.
mm/debug_vm_pgtable/savedwrite: Enable savedwrite test with
CONFIG_NUMA_BALANCING
mm/debug_vm_pgtable/THP: Mark the pte entry huge before using
set_pmd/pud_at
mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an
existing pte entry
mm/debug_vm_pgtable/thp: Use page table depost/withdraw with THP mm/debug_vm_pgtable/locks: Move non page table modifying test together mm/debug_vm_pgtable/locks: Take correct page table lock mm/debug_vm_pgtable/pmd_clear: Don't use pmd/pud_clear on pte entries mm/debug_vm_pgtable/hugetlb: Disable hugetlb test on ppc64 mm/debug_vm_pgtable: populate a pte entry before fetching it

Quoted 8 lines
 arch/powerpc/include/asm/book3s/64/pgtable.h |  29 +++-
 arch/powerpc/include/asm/nohash/pgtable.h    |   5 -
 arch/powerpc/mm/book3s64/pgtable.c           |   2 +-
 arch/powerpc/mm/pgtable.c                    |   5 -
 include/linux/io.h                           |  12 ++
 mm/debug_vm_pgtable.c                        | 151 +++++++++++--------
 6 files changed, 127 insertions(+), 77 deletions(-)

BTW I picked a wrong branch when sending this. Attaching the diff against what I want to send. pfn_pmd() no more updates _PAGE_PTE because that is handled by pmd_mkhuge().

arch/powerpc/mm/book3s64/pgtable.c +1 -1
index 3b4da7c63e28..e18ae50a275c 100644
@@ -141,7 +141,7 @@ pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot)
 	unsigned long pmdv;
 
 	pmdv = (pfn << PAGE_SHIFT) & PTE_RPN_MASK;
-	return __pmd(pmdv | pgprot_val(pgprot) | _PAGE_PTE);
+	return pmd_set_protbits(__pmd(pmdv), pgprot);
 }
 
 pmd_t mk_pmd(struct page *page, pgprot_t pgprot)
mm/debug_vm_pgtable.c +0 -0
index 7d9f8e1d790f..cad61d22f33a 100644

mm/debug_vm_pgtable.c static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) · line 235

Quoted 8 lines
mm/debug_vm_pgtable.c +1 -1
@@ -229,7 +229,7 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
 
 static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
 {
-	pmd_t pmd = pfn_pmd(pfn, prot);
+	pmd_t pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
 
 	if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
 		return;

Cover letter does not mention which branch or tag this series applies on. Just assumed it to be 5.9-rc1. Should the above changes be captured as a pre-requisite patch ?

Anyways, the series fails to be build on arm64.

A) Without CONFIG_TRANSPARENT_HUGEPAGE

mm/debug_vm_pgtable.c: In function ‘debug_vm_pgtable’:
mm/debug_vm_pgtable.c:1045:2: error: too many arguments to function ‘pmd_advanced_tests’
pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep); ^~~~~~~~~~~~~~~~~~
mm/debug_vm_pgtable.c:366:20: note: declared here
static void __init pmd_advanced_tests(struct mm_struct *mm,

                    ^~~~~~~~~~~~~~~~~~

B) As mentioned previously, this should be solved by including <linux/io.h>

mm/debug_vm_pgtable.c: In function ‘pmd_huge_tests’:
mm/debug_vm_pgtable.c:215:7: error: implicit declaration of function ‘arch_ioremap_pmd_supported’; did you mean ‘arch_disable_smp_support’? [-Werror=implicit-function-declaration]
if (!arch_ioremap_pmd_supported())

Quoted 7 lines
       ^~~~~~~~~~~~~~~~~~~~~~~~~~

Please make sure that the series builds on all enabled platforms i.e x86, arm64, ppc32, ppc64, arc, s390 along with selectively enabling/disabling all the features that make various #ifdefs in the test.

- Anshuman

Here is another build failure on x86.

mm/debug_vm_pgtable.c: In function ‘pud_advanced_tests’:
mm/debug_vm_pgtable.c:306:31: error: passing argument 1 of ‘pudp_huge_get_and_clear_full’ from incompatible pointer type [-Werror=incompatible-pointer-types]

  pudp_huge_get_and_clear_full(vma, vaddr, pudp, 1);
                               ^~~
In file included from ./include/linux/mm.h:33:0,
                 from ./include/linux/highmem.h:8,
                 from mm/debug_vm_pgtable.c:14:

./include/linux/pgtable.h:294:21: note: expected ‘struct mm_struct *’ but argument is of type ‘struct vm_area_struct *’
static inline pud_t pudp_huge_get_and_clear_full(struct mm_struct *mm,

#25 lore raw
In reply to Anshuman Khandual
Quoted 96 lines · Anshuman Khandual
Quoted 66 lines · Aneesh Kumar K.V
Quoted 32 lines · Aneesh Kumar K.V

This patch series includes fixes for debug_vm_pgtable test code so that they follow page table updates rules correctly. The first two patches introduce changes w.r.t ppc64. The patches are included in this series for completeness. We can merge them via ppc64 tree if required.

Hugetlb test is disabled on ppc64 because that needs larger change to satisfy page table update rules.

Changes from V1:
* Address review feedback
* drop test specific pfn_pte and pfn_pmd.
* Update ppc64 page table helper to add _PAGE_PTE

Aneesh Kumar K.V (13):
powerpc/mm: Add DEBUG_VM WARN for pmd_clear
powerpc/mm: Move setting pte specific flags to pfn_pte mm/debug_vm_pgtable/ppc64: Avoid setting top bits in radom value mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge
vmap support.
mm/debug_vm_pgtable/savedwrite: Enable savedwrite test with
CONFIG_NUMA_BALANCING
mm/debug_vm_pgtable/THP: Mark the pte entry huge before using
set_pmd/pud_at
mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an
existing pte entry
mm/debug_vm_pgtable/thp: Use page table depost/withdraw with THP mm/debug_vm_pgtable/locks: Move non page table modifying test together mm/debug_vm_pgtable/locks: Take correct page table lock mm/debug_vm_pgtable/pmd_clear: Don't use pmd/pud_clear on pte entries mm/debug_vm_pgtable/hugetlb: Disable hugetlb test on ppc64 mm/debug_vm_pgtable: populate a pte entry before fetching it

Quoted 8 lines
  arch/powerpc/include/asm/book3s/64/pgtable.h |  29 +++-
  arch/powerpc/include/asm/nohash/pgtable.h    |   5 -
  arch/powerpc/mm/book3s64/pgtable.c           |   2 +-
  arch/powerpc/mm/pgtable.c                    |   5 -
  include/linux/io.h                           |  12 ++
  mm/debug_vm_pgtable.c                        | 151 +++++++++++--------
  6 files changed, 127 insertions(+), 77 deletions(-)

BTW I picked a wrong branch when sending this. Attaching the diff against what I want to send. pfn_pmd() no more updates _PAGE_PTE because that is handled by pmd_mkhuge().

arch/powerpc/mm/book3s64/pgtable.c +1 -1
index 3b4da7c63e28..e18ae50a275c 100644
@@ -141,7 +141,7 @@ pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot)
  	unsigned long pmdv;
 
  	pmdv = (pfn << PAGE_SHIFT) & PTE_RPN_MASK;
-	return __pmd(pmdv | pgprot_val(pgprot) | _PAGE_PTE);
+	return pmd_set_protbits(__pmd(pmdv), pgprot);
  }
 
  pmd_t mk_pmd(struct page *page, pgprot_t pgprot)
mm/debug_vm_pgtable.c +0 -0
index 7d9f8e1d790f..cad61d22f33a 100644

mm/debug_vm_pgtable.c static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) · line 235

Quoted 8 lines
mm/debug_vm_pgtable.c +1 -1
@@ -229,7 +229,7 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
 
  static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
  {
-	pmd_t pmd = pfn_pmd(pfn, prot);
+	pmd_t pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
 
  	if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
  		return;

Cover letter does not mention which branch or tag this series applies on. Just assumed it to be 5.9-rc1. Should the above changes be captured as a pre-requisite patch ?

Anyways, the series fails to be build on arm64.

A) Without CONFIG_TRANSPARENT_HUGEPAGE

mm/debug_vm_pgtable.c: In function ‘debug_vm_pgtable’:
mm/debug_vm_pgtable.c:1045:2: error: too many arguments to function ‘pmd_advanced_tests’

   pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep);
   ^~~~~~~~~~~~~~~~~~
mm/debug_vm_pgtable.c:366:20: note: declared here
  static void __init pmd_advanced_tests(struct mm_struct *mm,
                     ^~~~~~~~~~~~~~~~~~

B) As mentioned previously, this should be solved by including <linux/io.h>

Quoted 9 lines

mm/debug_vm_pgtable.c: In function ‘pmd_huge_tests’:
mm/debug_vm_pgtable.c:215:7: error: implicit declaration of function ‘arch_ioremap_pmd_supported’; did you mean ‘arch_disable_smp_support’? [-Werror=implicit-function-declaration]
if (!arch_ioremap_pmd_supported())

        ^~~~~~~~~~~~~~~~~~~~~~~~~~

Please make sure that the series builds on all enabled platforms i.e x86, arm64, ppc32, ppc64, arc, s390 along with selectively enabling/disabling all the features that make various #ifdefs in the test.

I was hoping to get kernel test robot build report to verify that. But if you can help with that i have pushed a branch to github with reported build failure fixes.

https://github.com/kvaneesh/linux/tree/debug_vm_pgtable

I still haven't looked at the PMD_FOLDED feedback from Christophe because I am not sure i follow why we are checking for PMD folded there.

-aneesh

#26 lore raw
Quoted 103 lines · Aneesh Kumar K.V
Quoted 92 lines · Anshuman Khandual
Quoted 63 lines · Aneesh Kumar K.V
Quoted 32 lines · Aneesh Kumar K.V

This patch series includes fixes for debug_vm_pgtable test code so that they follow page table updates rules correctly. The first two patches introduce changes w.r.t ppc64. The patches are included in this series for completeness. We can merge them via ppc64 tree if required.

Hugetlb test is disabled on ppc64 because that needs larger change to satisfy page table update rules.

Changes from V1:
* Address review feedback
* drop test specific pfn_pte and pfn_pmd.
* Update ppc64 page table helper to add _PAGE_PTE

Aneesh Kumar K.V (13):
   powerpc/mm: Add DEBUG_VM WARN for pmd_clear
   powerpc/mm: Move setting pte specific flags to pfn_pte mm/debug_vm_pgtable/ppc64: Avoid setting top bits in radom value mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge vmap support.
   mm/debug_vm_pgtable/savedwrite: Enable savedwrite test with CONFIG_NUMA_BALANCING
   mm/debug_vm_pgtable/THP: Mark the pte entry huge before using set_pmd/pud_at
   mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an existing pte entry
   mm/debug_vm_pgtable/thp: Use page table depost/withdraw with THP mm/debug_vm_pgtable/locks: Move non page table modifying test together mm/debug_vm_pgtable/locks: Take correct page table lock mm/debug_vm_pgtable/pmd_clear: Don't use pmd/pud_clear on pte entries mm/debug_vm_pgtable/hugetlb: Disable hugetlb test on ppc64 mm/debug_vm_pgtable: populate a pte entry before fetching it

Quoted 8 lines

  arch/powerpc/include/asm/book3s/64/pgtable.h |  29 +++- arch/powerpc/include/asm/nohash/pgtable.h    |   5 - arch/powerpc/mm/book3s64/pgtable.c           |   2 +- arch/powerpc/mm/pgtable.c                    |   5 - include/linux/io.h                           |  12 ++ mm/debug_vm_pgtable.c                        | 151 +++++++++++--------

  6 files changed, 127 insertions(+), 77 deletions(-)

BTW I picked a wrong branch when sending this. Attaching the diff against what I want to send.  pfn_pmd() no more updates _PAGE_PTE because that is handled by pmd_mkhuge().

arch/powerpc/mm/book3s64/pgtable.c +1 -1
index 3b4da7c63e28..e18ae50a275c 100644
@@ -141,7 +141,7 @@ pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot)
       unsigned long pmdv;
         pmdv = (pfn << PAGE_SHIFT) & PTE_RPN_MASK;
-    return __pmd(pmdv | pgprot_val(pgprot) | _PAGE_PTE);
+    return pmd_set_protbits(__pmd(pmdv), pgprot);
   }
     pmd_t mk_pmd(struct page *page, pgprot_t pgprot)
mm/debug_vm_pgtable.c +0 -0
index 7d9f8e1d790f..cad61d22f33a 100644

mm/debug_vm_pgtable.c static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) · line 234

Quoted 7 lines
mm/debug_vm_pgtable.c +1 -1
@@ -229,7 +229,7 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
     static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
   {
-    pmd_t pmd = pfn_pmd(pfn, prot);
+    pmd_t pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
         if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
           return;
 

Cover letter does not mention which branch or tag this series applies on. Just assumed it to be 5.9-rc1. Should the above changes be captured as a pre-requisite patch ?

Anyways, the series fails to be build on arm64.

A) Without CONFIG_TRANSPARENT_HUGEPAGE

mm/debug_vm_pgtable.c: In function ‘debug_vm_pgtable’:
mm/debug_vm_pgtable.c:1045:2: error: too many arguments to function ‘pmd_advanced_tests’ pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep); ^~~~~~~~~~~~~~~~~~
mm/debug_vm_pgtable.c:366:20: note: declared here
  static void __init pmd_advanced_tests(struct mm_struct *mm, ^~~~~~~~~~~~~~~~~~

B) As mentioned previously, this should be solved by including <linux/io.h>

Quoted 9 lines

mm/debug_vm_pgtable.c: In function ‘pmd_huge_tests’:
mm/debug_vm_pgtable.c:215:7: error: implicit declaration of function ‘arch_ioremap_pmd_supported’; did you mean ‘arch_disable_smp_support’? [-Werror=implicit-function-declaration] if (!arch_ioremap_pmd_supported())
        ^~~~~~~~~~~~~~~~~~~~~~~~~~

Please make sure that the series builds on all enabled platforms i.e x86, arm64, ppc32, ppc64, arc, s390 along with selectively enabling/disabling all the features that make various #ifdefs in the test.

Quoted 5 lines

I was hoping to get kernel test robot build report to verify that. But if you can help with that i have pushed a branch to github with reported build failure fixes.

https://github.com/kvaneesh/linux/tree/debug_vm_pgtable

I still haven't looked at the PMD_FOLDED feedback from Christophe because I am not sure i follow why we are checking for PMD folded there.

If this series does not build on existing enabled platforms, wondering how effective the review could be, assuming that things would need to change again to fix those build failures on various platforms. Getting this to build here is essential, as not all page table constructs are available across these platforms. Hence wondering, it might be better if you could resend the series after fixing build issues.

#27 lore raw
Quoted 110 lines · Anshuman Khandual
Quoted 107 lines · Aneesh Kumar K.V
Quoted 92 lines · Anshuman Khandual
Quoted 63 lines · Aneesh Kumar K.V
Quoted 32 lines · Aneesh Kumar K.V

This patch series includes fixes for debug_vm_pgtable test code so that they follow page table updates rules correctly. The first two patches introduce changes w.r.t ppc64. The patches are included in this series for completeness. We can merge them via ppc64 tree if required.

Hugetlb test is disabled on ppc64 because that needs larger change to satisfy page table update rules.

Changes from V1:
* Address review feedback
* drop test specific pfn_pte and pfn_pmd.
* Update ppc64 page table helper to add _PAGE_PTE

Aneesh Kumar K.V (13):
   powerpc/mm: Add DEBUG_VM WARN for pmd_clear
   powerpc/mm: Move setting pte specific flags to pfn_pte mm/debug_vm_pgtable/ppc64: Avoid setting top bits in radom value mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge vmap support.
   mm/debug_vm_pgtable/savedwrite: Enable savedwrite test with CONFIG_NUMA_BALANCING
   mm/debug_vm_pgtable/THP: Mark the pte entry huge before using set_pmd/pud_at
   mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an existing pte entry
   mm/debug_vm_pgtable/thp: Use page table depost/withdraw with THP mm/debug_vm_pgtable/locks: Move non page table modifying test together mm/debug_vm_pgtable/locks: Take correct page table lock mm/debug_vm_pgtable/pmd_clear: Don't use pmd/pud_clear on pte entries mm/debug_vm_pgtable/hugetlb: Disable hugetlb test on ppc64 mm/debug_vm_pgtable: populate a pte entry before fetching it

Quoted 8 lines
   arch/powerpc/include/asm/book3s/64/pgtable.h |  29 +++-
   arch/powerpc/include/asm/nohash/pgtable.h    |   5 -
   arch/powerpc/mm/book3s64/pgtable.c           |   2 +-
   arch/powerpc/mm/pgtable.c                    |   5 -
   include/linux/io.h                           |  12 ++
   mm/debug_vm_pgtable.c                        | 151 +++++++++++--------
   6 files changed, 127 insertions(+), 77 deletions(-)

BTW I picked a wrong branch when sending this. Attaching the diff against what I want to send.  pfn_pmd() no more updates _PAGE_PTE because that is handled by pmd_mkhuge().

arch/powerpc/mm/book3s64/pgtable.c +1 -1
index 3b4da7c63e28..e18ae50a275c 100644
@@ -141,7 +141,7 @@ pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot)
       unsigned long pmdv;
         pmdv = (pfn << PAGE_SHIFT) & PTE_RPN_MASK;
-    return __pmd(pmdv | pgprot_val(pgprot) | _PAGE_PTE);
+    return pmd_set_protbits(__pmd(pmdv), pgprot);
   }
     pmd_t mk_pmd(struct page *page, pgprot_t pgprot)
mm/debug_vm_pgtable.c +0 -0
index 7d9f8e1d790f..cad61d22f33a 100644

mm/debug_vm_pgtable.c static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) · line 234

Quoted 7 lines
mm/debug_vm_pgtable.c +1 -1
@@ -229,7 +229,7 @@ static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot)
     static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot)
   {
-    pmd_t pmd = pfn_pmd(pfn, prot);
+    pmd_t pmd = pmd_mkhuge(pfn_pmd(pfn, prot));
         if (!IS_ENABLED(CONFIG_NUMA_BALANCING))
           return;
 

Cover letter does not mention which branch or tag this series applies on. Just assumed it to be 5.9-rc1. Should the above changes be captured as a pre-requisite patch ?

Anyways, the series fails to be build on arm64.

A) Without CONFIG_TRANSPARENT_HUGEPAGE

mm/debug_vm_pgtable.c: In function ‘debug_vm_pgtable’:
mm/debug_vm_pgtable.c:1045:2: error: too many arguments to function ‘pmd_advanced_tests’
   pmd_advanced_tests(mm, vma, pmdp, pmd_aligned, vaddr, prot, saved_ptep); ^~~~~~~~~~~~~~~~~~
mm/debug_vm_pgtable.c:366:20: note: declared here
  static void __init pmd_advanced_tests(struct mm_struct *mm, ^~~~~~~~~~~~~~~~~~

B) As mentioned previously, this should be solved by including <linux/io.h>

Quoted 9 lines

mm/debug_vm_pgtable.c: In function ‘pmd_huge_tests’:
mm/debug_vm_pgtable.c:215:7: error: implicit declaration of function ‘arch_ioremap_pmd_supported’; did you mean ‘arch_disable_smp_support’? [-Werror=implicit-function-declaration]
   if (!arch_ioremap_pmd_supported())
        ^~~~~~~~~~~~~~~~~~~~~~~~~~

Please make sure that the series builds on all enabled platforms i.e x86, arm64, ppc32, ppc64, arc, s390 along with selectively enabling/disabling all the features that make various #ifdefs in the test.

I was hoping to get kernel test robot build report to verify that. But if you can help with that i have pushed a branch to github with reported build failure fixes.

https://github.com/kvaneesh/linux/tree/debug_vm_pgtable

Quoted 9 lines
Quoted 1 line

I still haven't looked at the PMD_FOLDED feedback from Christophe because I am not sure i follow why we are checking for PMD folded there.

If this series does not build on existing enabled platforms, wondering how effective the review could be, assuming that things would need to change again to fix those build failures on various platforms. Getting this to build here is essential, as not all page table constructs are available across these platforms. Hence wondering, it might be better if you could resend the series after fixing build issues.

Sure. I am hoping kernel test robot will pick this up. I did an x86 and about 19 different ppc config build with the series. The git tree above was pushed with that. Considering you authored the change i am wondering if you could help with checking other architecture (may be atleast arm variant)

-aneesh

#28 lore raw
Quoted 3 lines

Sure. I am hoping kernel test robot will pick this up. I did an x86 and about 19 different ppc config build with the series. The git tree above was pushed with that. Considering you authored the change i am wondering if you could help with checking other architecture (may be atleast arm variant)

I updated the tree after a defconfig build on arch/arm64/s390/x86. I will not be able to boot test them.

Can you help with boot testing on arm?

-aneesh

#29 lore raw
Quoted 7 lines · Aneesh Kumar K.V
Quoted 3 lines

Sure. I am hoping kernel test robot will pick this up. I did an x86 and about 19 different ppc config build with the series. The git tree above was pushed with that. Considering you authored the change i am wondering if you could help with checking other architecture (may be atleast arm variant)

I updated the tree after a defconfig build on arch/arm64/s390/x86.  I will not be able to boot test them.

Can you help with boot testing on arm?

Did not see any obvious problem.

#30 lore raw
In reply to Aneesh Kumar K.V
Quoted 32 lines · Aneesh Kumar K.V

This patch series includes fixes for debug_vm_pgtable test code so that they follow page table updates rules correctly. The first two patches introduce changes w.r.t ppc64. The patches are included in this series for completeness. We can merge them via ppc64 tree if required.

Hugetlb test is disabled on ppc64 because that needs larger change to satisfy page table update rules.

Changes from V1:
* Address review feedback
* drop test specific pfn_pte and pfn_pmd.
* Update ppc64 page table helper to add _PAGE_PTE

Aneesh Kumar K.V (13):
powerpc/mm: Add DEBUG_VM WARN for pmd_clear
powerpc/mm: Move setting pte specific flags to pfn_pte mm/debug_vm_pgtable/ppc64: Avoid setting top bits in radom value mm/debug_vm_pgtables/hugevmap: Use the arch helper to identify huge
vmap support.
mm/debug_vm_pgtable/savedwrite: Enable savedwrite test with
CONFIG_NUMA_BALANCING
mm/debug_vm_pgtable/THP: Mark the pte entry huge before using
set_pmd/pud_at
mm/debug_vm_pgtable/set_pte/pmd/pud: Don't use set_*_at to update an
existing pte entry
mm/debug_vm_pgtable/thp: Use page table depost/withdraw with THP mm/debug_vm_pgtable/locks: Move non page table modifying test together mm/debug_vm_pgtable/locks: Take correct page table lock mm/debug_vm_pgtable/pmd_clear: Don't use pmd/pud_clear on pte entries mm/debug_vm_pgtable/hugetlb: Disable hugetlb test on ppc64 mm/debug_vm_pgtable: populate a pte entry before fetching it

Quoted 8 lines
 arch/powerpc/include/asm/book3s/64/pgtable.h |  29 +++-
 arch/powerpc/include/asm/nohash/pgtable.h    |   5 -
 arch/powerpc/mm/book3s64/pgtable.c           |   2 +-
 arch/powerpc/mm/pgtable.c                    |   5 -
 include/linux/io.h                           |  12 ++
 mm/debug_vm_pgtable.c                        | 151 +++++++++++--------
 6 files changed, 127 insertions(+), 77 deletions(-)

Changes proposed here will impact other enabled platforms as well. Adding the following folks and mailing lists, and hoping to get a broader review and test coverage. Please do include them in the next iteration as well.

+ [email protected]
+ [email protected]
+ [email protected]
+ [email protected]
+ [email protected]

+ Gerald Schaefer <[email protected]>
+ Christophe Leroy <[email protected]>
+ Christophe Leroy <[email protected]>
+ Vineet Gupta <[email protected]>
+ Mike Rapoport <[email protected]>
+ Qian Cai <[email protected]>

#31 lore raw

Le 21/08/2020 à 10:51, Anshuman Khandual a écrit :

Quoted 17 lines · Aneesh Kumar K.V
Quoted 8 lines

This patch series includes fixes for debug_vm_pgtable test code so that they follow page table updates rules correctly. The first two patches introduce changes w.r.t ppc64. The patches are included in this series for completeness. We can merge them via ppc64 tree if required.

Hugetlb test is disabled on ppc64 because that needs larger change to satisfy page table update rules.

Changes proposed here will impact other enabled platforms as well. Adding the following folks and mailing lists, and hoping to get a broader review and test coverage. Please do include them in the next iteration as well.

Quoted 8 lines

+ [email protected]
+ [email protected]
+ [email protected]
+ [email protected]
+ [email protected]

+ Gerald Schaefer <[email protected]>
+ Christophe Leroy <[email protected]>

Please don't use anymore the above address. Only use the one below.

Quoted 5 lines

+ Christophe Leroy <[email protected]>
+ Vineet Gupta <[email protected]>
+ Mike Rapoport <[email protected]>
+ Qian Cai <[email protected]>

Thanks
Christophe

#32 lore raw
Quoted 26 lines · Christophe Leroy

Le 21/08/2020 à 10:51, Anshuman Khandual a écrit :

Quoted 23 lines · Aneesh Kumar K.V
Quoted 8 lines

This patch series includes fixes for debug_vm_pgtable test code so that they follow page table updates rules correctly. The first two patches introduce changes w.r.t ppc64. The patches are included in this series for completeness. We can merge them via ppc64 tree if required.

Hugetlb test is disabled on ppc64 because that needs larger change to satisfy page table update rules.

Changes proposed here will impact other enabled platforms as well. Adding the following folks and mailing lists, and hoping to get a broader review and test coverage. Please do include them in the next iteration as well.

+ [email protected]
+ [email protected]
+ [email protected]
+ [email protected]
+ [email protected]

Quoted 6 lines
Quoted 2 lines

+ Gerald Schaefer <[email protected]>
+ Christophe Leroy <[email protected]>

Please don't use anymore the above address. Only use the one below.

Quoted 1 line

+ Christophe Leroy <[email protected]>

Sure, noted.

Quoted 9 lines
Quoted 4 lines

+ Vineet Gupta <[email protected]>
+ Mike Rapoport <[email protected]>
+ Qian Cai <[email protected]>

Thanks
Christophe