Skip to contentSkip to conversation

eldie generated code for folded p4d/pud

PATCH Superseded Vineet Gupta 4 participants 11 messages Oct 15, 2019 – Oct 16, 2019

Subsystems (5)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover eldie generated code for folded p4d/pud

Hi,

This series came out of seemingly naive exceursion into understanding/removing __ARCH_USE_5LEVEL_HACK from ARC port. With removal of 5LEVEL_HACK some extraneous code was bein generated

| bloat-o-meter2 vmlinux-[AB]*
| add/remove: 0/0 grow/shrink: 3/0 up/down: 130/0 (130)
| function                                     old     new   delta
| free_pgd_range                               548     660    +112
| p4d_clear_bad                                  2      20     +18

Which the patches here elides (for folded p4d/pud/pmd)

| bloat-o-meter2 vmlinux-[BF]*
| add/remove: 0/2 grow/shrink: 0/1 up/down: 0/-386 (-386)
| function                                     old     new   delta
| pud_clear_bad                                 20       -     -20
| p4d_clear_bad                                 20       -     -20
| free_pgd_range                               660     314    -346

The code savings are not a whole lot, but still worthwhile IMHO.

Please review, test and apply. It seems to survive my usual battery of multibench, hakcbench etc.

Thx,
-Vineet

---
Changes since v1 [1]
- Per Linus Sugestion remvoed the extra ifdey'ery (hence not accumulating Kirill's ACks)
- Added the RFC patch for pmd_free_tlb() after discussions [2]
- Also throwing in the ARC patch which started this all (so we get the full context of patchset) - I'm ok if this goes via mm tree, should be non contentious and can drop this too if Andrew thinks otherwise

[1] http://lists.infradead.org/pipermail/linux-snps-arc/2019-October/006263.html
[2] http://lists.infradead.org/pipermail/linux-snps-arc/2019-October/006277.html

---

Vineet Gupta (5):
ARC: mm: remove __ARCH_USE_5LEVEL_HACK asm-generic/tlb: stub out pud_free_tlb() if nopud ... asm-generic/tlb: stub out p4d_free_tlb() if nop4d ... asm-generic/tlb: stub out pmd_free_tlb() if nopmd asm-generic/mm: stub out p{4,d}d_clear_bad() if

    __PAGETABLE_P{4,u}D_FOLDED
 arch/arc/include/asm/pgtable.h      |  1 -
 arch/arc/mm/fault.c                 | 10 ++++++++--
 arch/arc/mm/highmem.c               |  4 +++-
 include/asm-generic/4level-fixup.h  |  1 -
 include/asm-generic/5level-fixup.h  |  1 -
 include/asm-generic/pgtable-nop4d.h |  2 +-
 include/asm-generic/pgtable-nopmd.h |  2 +-
 include/asm-generic/pgtable-nopud.h |  2 +-
 include/asm-generic/pgtable.h       | 11 +++++++++++
 include/asm-generic/tlb.h           |  4 ----
 mm/pgtable-generic.c                |  8 ++++++++
 11 files changed, 33 insertions(+), 13 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH v2 1/5 ARC: mm: remove __ARCH_USE_5LEVEL_HACK

Add the intermedivmlinux-A-baseline vmlinux-B-elide-ARCH_USE_5LEVEL_HACK

This is a non-functional change anyways since ARC has software page walker with 2 lookup levels (pgd -> pte)

There is slight code bloat due to pulling in needless p*d_free_tlb() macros which needs to be addressed seperately.

| bloat-o-meter2 vmlinux-with-5LEVEL_HACK vmlinux-patched
| add/remove: 0/0 grow/shrink: 2/0 up/down: 128/0 (128)
| function                                     old     new   delta
| free_pgd_range                               546     656    +110
| p4d_clear_bad                                  2      20     +18
| Total: Before=4137148, After=4137276, chg 0.000000%
Cc Kirill A. Shutemov <[email protected]>
---
 arch/arc/include/asm/pgtable.h |  1 -
 arch/arc/mm/fault.c            | 10 ++++++++--
 arch/arc/mm/highmem.c          |  4 +++-
 3 files changed, 11 insertions(+), 4 deletions(-)
arch/arc/include/asm/pgtable.h +0 -1
index 7addd0301c51..b917b596f7fb 100644
@@ -33,7 +33,6 @@
 #define _ASM_ARC_PGTABLE_H
 
 #include <linux/bits.h>
-#define __ARCH_USE_5LEVEL_HACK
 #include <asm-generic/pgtable-nopmd.h>
 #include <asm/page.h>
 #include <asm/mmu.h>	/* to propagate CONFIG_ARC_MMU_VER <n> */
arch/arc/mm/fault.c +8 -2
index 3861543b66a0..fb86bc3e9b35 100644
@@ -30,6 +30,7 @@ noinline static int handle_kernel_vaddr_fault(unsigned long address)
 	 * with the 'reference' page table.
 	 */
 	pgd_t *pgd, *pgd_k;
+	p4d_t *p4d, *p4d_k;
 	pud_t *pud, *pud_k;
 	pmd_t *pmd, *pmd_k;
 
@@ -39,8 +40,13 @@ noinline static int handle_kernel_vaddr_fault(unsigned long address)
 	if (!pgd_present(*pgd_k))
 		goto bad_area;
 
-	pud = pud_offset(pgd, address);
-	pud_k = pud_offset(pgd_k, address);
+	p4d = p4d_offset(pgd, address);
+	p4d_k = p4d_offset(pgd_k, address);
+	if (!p4d_present(*p4d_k))
+		goto bad_area;
+
+	pud = pud_offset(p4d, address);
+	pud_k = pud_offset(p4d_k, address);
 	if (!pud_present(*pud_k))
 		goto bad_area;
 
arch/arc/mm/highmem.c +3 -1
index a4856bfaedf3..fc8849e4f72e 100644
@@ -111,12 +111,14 @@ EXPORT_SYMBOL(__kunmap_atomic);
 static noinline pte_t * __init alloc_kmap_pgtable(unsigned long kvaddr)
 {
 	pgd_t *pgd_k;
+	p4d_t *p4d_k;
 	pud_t *pud_k;
 	pmd_t *pmd_k;
 	pte_t *pte_k;
 
 	pgd_k = pgd_offset_k(kvaddr);
-	pud_k = pud_offset(pgd_k, kvaddr);
+	p4d_k = p4d_offset(pgd_k, kvaddr);
+	pud_k = pud_offset(p4d_k, kvaddr);
 	pmd_k = pmd_offset(pud_k, kvaddr);
 
 	pte_k = (pte_t *)memblock_alloc_low(PAGE_SIZE, PAGE_SIZE);
Signature · 1 line
2.20.1
#2 lore raw

Re: [PATCH v2 1/5] ARC: mm: remove __ARCH_USE_5LEVEL_HACK

Quoted 3 lines · Vineet Gupta

This is a non-functional change anyways since ARC has software page walker with 2 lookup levels (pgd -> pte)

Could we encourage other architectures to do the same, and get rid of all uses of __ARCH_USE_5LEVEL_HACK?

Linus

#3 lore raw

Re: [PATCH v2 1/5] ARC: mm: remove __ARCH_USE_5LEVEL_HACK

Quoted 7 lines · Linus Torvalds
Quoted 2 lines · Vineet Gupta

This is a non-functional change anyways since ARC has software page walker with 2 lookup levels (pgd -> pte)

Could we encourage other architectures to do the same, and get rid of all uses of __ARCH_USE_5LEVEL_HACK?

IMHO this should have been done at the onset. The actual changes don't seem that difficult, just need to add the missing p4d calls although I can sympathize with hassles of coordinating/building/testing/yadi yada cross arch.

OTOH, the [45]LEVEL_HACK seem like a nice way to "fold" the levels: the skipped/folded level vanishes completely. Among others it does things like

     #define p4d_t	pgd_t	

On ARC 2-level code:

free_pte_range

	pmd_pgtable(*pmd);

	    ((((((*pmd).pud).pgd))) & PAGE_MASK));	     <-- 5LEVEL_HACK
			vs.
	    ((((((((*pmd).pud).p4d).pgd)))) & PAGE_MASK ));  <-- w/o 5LEVEL_HACK
	
	pmd_clear(pmd);

	    *(pmd)).pud).pgd)))) = 0
			vs.
	    *(pmd)).pud).p4d).pgd)))) = 0

So we may not be able to fix all he historical misgivigs, but this might alleviate the pain a bit. I'll try to dabble a bit.

Thx for taking a look and te ACKs.
-Vineet

#4 lore raw

PATCH v2 2/5 asm-generic/tlb: stub out pud_free_tlb() if nopud ...

In reply to Vineet Gupta

... independent of __ARCH_HAS_4LEVEL_HACK

This came up when removing __ARCH_HAS_5LEVEL_HACK for ARC as code bloat from this routine which is not required in a 2-level paging setup

Note: The primary change is alternate defines for pud_free_tlb() but while here removed empty stubs for __pud_free_tlb as, which is only called from pud_free_tlb(), being conditionalized already

| bloat-o-meter2 vmlinux-B-elide-ARCH_USE_5LEVEL_HACK vmlinux-C-elide-pud_free_tlb
| add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-104 (-104)
| function                                     old     new   delta
| free_pgd_range                               656     552    -104
| Total: Before=4137276, After=4137172, chg -1.000000%
---
 include/asm-generic/4level-fixup.h  | 1 -
 include/asm-generic/pgtable-nopud.h | 2 +-
 include/asm-generic/tlb.h           | 2 --
 3 files changed, 1 insertion(+), 4 deletions(-)
include/asm-generic/4level-fixup.h +0 -1
index e3667c9a33a5..c86cf7cb4bba 100644
@@ -30,7 +30,6 @@
 #undef pud_free_tlb
 #define pud_free_tlb(tlb, x, addr)	do { } while (0)
 #define pud_free(mm, x)			do { } while (0)
-#define __pud_free_tlb(tlb, x, addr)	do { } while (0)
 
 #undef  pud_addr_end
 #define pud_addr_end(addr, end)		(end)
include/asm-generic/pgtable-nopud.h +1 -1
index c77a1d301155..d3776cb494c0 100644
@@ -59,7 +59,7 @@ static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address)
  */
 #define pud_alloc_one(mm, address)		NULL
 #define pud_free(mm, x)				do { } while (0)
-#define __pud_free_tlb(tlb, x, a)		do { } while (0)
+#define pud_free_tlb(tlb, x, a)		        do { } while (0)
 
 #undef  pud_addr_end
 #define pud_addr_end(addr, end)			(end)
include/asm-generic/tlb.h +0 -2
index 04c0644006fd..5e0c2d01e656 100644
@@ -584,7 +584,6 @@ static inline void tlb_end_vma(struct mmu_gather *tlb, struct vm_area_struct *vm
 	} while (0)
 #endif
 
-#ifndef __ARCH_HAS_4LEVEL_HACK
 #ifndef pud_free_tlb
 #define pud_free_tlb(tlb, pudp, address)			\
 	do {							\
@@ -594,7 +593,6 @@ static inline void tlb_end_vma(struct mmu_gather *tlb, struct vm_area_struct *vm
 		__pud_free_tlb(tlb, pudp, address);		\
 	} while (0)
 #endif
-#endif
 
 #ifndef __ARCH_HAS_5LEVEL_HACK
 #ifndef p4d_free_tlb
Signature · 1 line
2.20.1
#5 lore raw

Re: [PATCH v2 2/5] asm-generic/tlb: stub out pud_free_tlb() if nopud ...

Quoted 3 lines · Vineet Gupta

This came up when removing __ARCH_HAS_5LEVEL_HACK for ARC as code bloat from this routine which is not required in a 2-level paging setup

Ack, looks good.

Linus

#6 lore raw

PATCH v2 3/5 asm-generic/tlb: stub out p4d_free_tlb() if nop4d ...

In reply to Vineet Gupta

... independent of __ARCH_HAS_5LEVEL_HACK

This came up when removing __ARCH_HAS_5LEVEL_HACK for ARC as code bloat from this routine not required in a 2-level paging setup

| bloat-o-meter2 vmlinux-C-elide-pud_free_tlb vmlinux-D-elide-p4d_free_tlb
| add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-104 (-104)
| function                                     old     new   delta
| free_pgd_range                               552     422    -130
| Total: Before=4137172, After=4137042, chg -1.000000%
---
 include/asm-generic/5level-fixup.h  | 1 -
 include/asm-generic/pgtable-nop4d.h | 2 +-
 include/asm-generic/tlb.h           | 2 --
 3 files changed, 1 insertion(+), 4 deletions(-)
include/asm-generic/5level-fixup.h +0 -1
index f6947da70d71..4c74b1c1d13b 100644
@@ -51,7 +51,6 @@ static inline int p4d_present(p4d_t p4d)
 #undef p4d_free_tlb
 #define p4d_free_tlb(tlb, x, addr)	do { } while (0)
 #define p4d_free(mm, x)			do { } while (0)
-#define __p4d_free_tlb(tlb, x, addr)	do { } while (0)
 
 #undef  p4d_addr_end
 #define p4d_addr_end(addr, end)		(end)
include/asm-generic/pgtable-nop4d.h +1 -1
index aebab905e6cd..ce2cbb3c380f 100644
@@ -50,7 +50,7 @@ static inline p4d_t *p4d_offset(pgd_t *pgd, unsigned long address)
  */
 #define p4d_alloc_one(mm, address)		NULL
 #define p4d_free(mm, x)				do { } while (0)
-#define __p4d_free_tlb(tlb, x, a)		do { } while (0)
+#define p4d_free_tlb(tlb, x, a)			do { } while (0)
 
 #undef  p4d_addr_end
 #define p4d_addr_end(addr, end)			(end)
include/asm-generic/tlb.h +0 -2
index 5e0c2d01e656..05dddc17522b 100644
@@ -594,7 +594,6 @@ static inline void tlb_end_vma(struct mmu_gather *tlb, struct vm_area_struct *vm
 	} while (0)
 #endif
 
-#ifndef __ARCH_HAS_5LEVEL_HACK
 #ifndef p4d_free_tlb
 #define p4d_free_tlb(tlb, pudp, address)			\
 	do {							\
@@ -603,7 +602,6 @@ static inline void tlb_end_vma(struct mmu_gather *tlb, struct vm_area_struct *vm
 		__p4d_free_tlb(tlb, pudp, address);		\
 	} while (0)
 #endif
-#endif
 
 #endif /* CONFIG_MMU */
 
Signature · 1 line
2.20.1
#7 lore raw

Re: [PATCH v2 3/5] asm-generic/tlb: stub out p4d_free_tlb() if nop4d ...

Quoted 3 lines · Vineet Gupta

This came up when removing __ARCH_HAS_5LEVEL_HACK for ARC as code bloat from this routine not required in a 2-level paging setup

Similarly acked,

Linus

#8 lore raw

PATCH v2 4/5 asm-generic/tlb: stub out pmd_free_tlb() if nopmd

In reply to Vineet Gupta

Note that pmd routine folding can be tricky as even in 2-level setup (where pmd is folded) most pmd routines refer to upper levels. This one can surely be elided however.

| bloat-o-meter2 vmlinux-E-elide-p?d_clear_bad vmlinux-F-elide-pmd_free_tlb
| add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-112 (-112)
| function                                     old     new   delta
| free_pgd_range                               422     310    -112
| Total: Before=4137042, After=4136930, chg -1.000000%
---
 include/asm-generic/pgtable-nopmd.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/asm-generic/pgtable-nopmd.h +1 -1
index b85b8271a73d..0d9b28cba16d 100644
@@ -60,7 +60,7 @@ static inline pmd_t * pmd_offset(pud_t * pud, unsigned long address)
 static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
 {
 }
-#define __pmd_free_tlb(tlb, x, a)		do { } while (0)
+#define pmd_free_tlb(tlb, x, a)		do { } while (0)
 
 #undef  pmd_addr_end
 #define pmd_addr_end(addr, end)			(end)
Signature · 1 line
2.20.1
#9 lore raw

PATCH v2 5/5 asm-generic/mm: stub out p{4,d}d_clear_bad() if __PAGETABLE_P{4,u}D_FOLDED

In reply to Vineet Gupta

This removes the code for 2 level paging as seen on ARC

| bloat-o-meter2 vmlinux-D-elide-p4d_free_tlb vmlinux-E-elide-p?d_clear_bad
| add/remove: 0/2 grow/shrink: 0/0 up/down: 0/-40 (-40)
| function                                     old     new   delta
| pud_clear_bad                                 20       -     -20
| p4d_clear_bad                                 20       -     -20
| Total: Before=4136930, After=4136890, chg -1.000000%
---
 include/asm-generic/pgtable.h | 11 +++++++++++
 mm/pgtable-generic.c          |  8 ++++++++
 2 files changed, 19 insertions(+)
include/asm-generic/pgtable.h +11 -0
index 818691846c90..9cdcbc7c0b7b 100644
@@ -558,8 +558,19 @@ static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot)
  * Do the tests inline, but report and clear the bad entry in mm/memory.c.
  */
 void pgd_clear_bad(pgd_t *);
+
+#ifndef __PAGETABLE_P4D_FOLDED
 void p4d_clear_bad(p4d_t *);
+#else
+#define p4d_clear_bad(p4d)        do { } while (0)
+#endif
+
+#ifndef __PAGETABLE_PUD_FOLDED
 void pud_clear_bad(pud_t *);
+#else
+#define pud_clear_bad(p4d)        do { } while (0)
+#endif
+
 void pmd_clear_bad(pmd_t *);
 
 static inline int pgd_none_or_clear_bad(pgd_t *pgd)
mm/pgtable-generic.c +8 -0
index 532c29276fce..a5edddc3846a 100644
@@ -24,18 +24,26 @@ void pgd_clear_bad(pgd_t *pgd)
 	pgd_clear(pgd);
 }
 
+#ifndef __PAGETABLE_P4D_FOLDED
 void p4d_clear_bad(p4d_t *p4d)
 {
 	p4d_ERROR(*p4d);
 	p4d_clear(p4d);
 }
+#endif
 
+#ifndef __PAGETABLE_PUD_FOLDED
 void pud_clear_bad(pud_t *pud)
 {
 	pud_ERROR(*pud);
 	pud_clear(pud);
 }
+#endif
 
+/*
+ * Note that below can't be stubed out for nopmd case:
+ * pmd folding is special and typically pmd_* macros refet to upper level
+ */
 void pmd_clear_bad(pmd_t *pmd)
 {
 	pmd_ERROR(*pmd);
Signature · 1 line
2.20.1
#10 lore raw

Re: [PATCH v2 5/5] asm-generic/mm: stub out p{4,d}d_clear_bad() if __PAGETABLE_P{4,u}D_FOLDED

There is couple of typos in the subject of the patch. It has to be

	[PATCH v2 5/5] asm-generic/mm: stub out p{4,u}d_clear_bad() if __PAGETABLE_P{4,U}D_FOLDED

Otherwise the patchset looks good to me. You can use my ACK for all patches.

Signature · 1 line
 Kirill A. Shutemov