eldie generated code for folded p4d/pud
Subsystems (5)
v2 cover eldie generated code for folded p4d/pud
Hi,
This series came out of seemingly naive exceursion into understanding/removing __ARCH_USE_5LEVEL_HACK from ARC port. With removal of 5LEVEL_HACK some extraneous code was bein generated
| bloat-o-meter2 vmlinux-[AB]* | add/remove: 0/0 grow/shrink: 3/0 up/down: 130/0 (130) | function old new delta | free_pgd_range 548 660 +112 | p4d_clear_bad 2 20 +18
Which the patches here elides (for folded p4d/pud/pmd)
| bloat-o-meter2 vmlinux-[BF]* | add/remove: 0/2 grow/shrink: 0/1 up/down: 0/-386 (-386) | function old new delta | pud_clear_bad 20 - -20 | p4d_clear_bad 20 - -20 | free_pgd_range 660 314 -346
The code savings are not a whole lot, but still worthwhile IMHO.
Please review, test and apply. It seems to survive my usual battery of multibench, hakcbench etc.
Thx,
-Vineet
---
Changes since v1 [1]
- Per Linus Sugestion remvoed the extra ifdey'ery (hence not accumulating Kirill's ACks)
- Added the RFC patch for pmd_free_tlb() after discussions [2]
- Also throwing in the ARC patch which started this all (so we get the full context of patchset) - I'm ok if this goes via mm tree, should be non contentious and can drop this too if Andrew thinks otherwise
[1] http://lists.infradead.org/pipermail/linux-snps-arc/2019-October/006263.html
[2] http://lists.infradead.org/pipermail/linux-snps-arc/2019-October/006277.html
---
Vineet Gupta (5):
ARC: mm: remove __ARCH_USE_5LEVEL_HACK asm-generic/tlb: stub out pud_free_tlb() if nopud ... asm-generic/tlb: stub out p4d_free_tlb() if nop4d ... asm-generic/tlb: stub out pmd_free_tlb() if nopmd asm-generic/mm: stub out p{4,d}d_clear_bad() if
__PAGETABLE_P{4,u}D_FOLDEDarch/arc/include/asm/pgtable.h | 1 - arch/arc/mm/fault.c | 10 ++++++++-- arch/arc/mm/highmem.c | 4 +++- include/asm-generic/4level-fixup.h | 1 - include/asm-generic/5level-fixup.h | 1 - include/asm-generic/pgtable-nop4d.h | 2 +- include/asm-generic/pgtable-nopmd.h | 2 +- include/asm-generic/pgtable-nopud.h | 2 +- include/asm-generic/pgtable.h | 11 +++++++++++ include/asm-generic/tlb.h | 4 ---- mm/pgtable-generic.c | 8 ++++++++ 11 files changed, 33 insertions(+), 13 deletions(-)
Signature · 1 line
2.20.1
PATCH v2 1/5 ARC: mm: remove __ARCH_USE_5LEVEL_HACK
Add the intermedivmlinux-A-baseline vmlinux-B-elide-ARCH_USE_5LEVEL_HACK
This is a non-functional change anyways since ARC has software page walker with 2 lookup levels (pgd -> pte)
There is slight code bloat due to pulling in needless p*d_free_tlb() macros which needs to be addressed seperately.
| bloat-o-meter2 vmlinux-with-5LEVEL_HACK vmlinux-patched | add/remove: 0/0 grow/shrink: 2/0 up/down: 128/0 (128) | function old new delta | free_pgd_range 546 656 +110 | p4d_clear_bad 2 20 +18 | Total: Before=4137148, After=4137276, chg 0.000000%
--- arch/arc/include/asm/pgtable.h | 1 - arch/arc/mm/fault.c | 10 ++++++++-- arch/arc/mm/highmem.c | 4 +++- 3 files changed, 11 insertions(+), 4 deletions(-)
arch/arc/include/asm/pgtable.h +0 -1
@@ -33,7 +33,6 @@#define _ASM_ARC_PGTABLE_H#include <linux/bits.h> -#define __ARCH_USE_5LEVEL_HACK#include <asm-generic/pgtable-nopmd.h> #include <asm/page.h> #include <asm/mmu.h> /* to propagate CONFIG_ARC_MMU_VER <n> */
arch/arc/mm/fault.c +8 -2
@@ -30,6 +30,7 @@ noinline static int handle_kernel_vaddr_fault(unsigned long address)* with the 'reference' page table. */ pgd_t *pgd, *pgd_k; +p4d_t *p4d, *p4d_k;pud_t *pud, *pud_k;pmd_t *pmd, *pmd_k;
@@ -39,8 +40,13 @@ noinline static int handle_kernel_vaddr_fault(unsigned long address)if (!pgd_present (*pgd_k)) goto bad_area; - pud =pud_offset (pgd, address); - pud_k =pud_offset (pgd_k, address); + p4d =p4d_offset (pgd, address); + p4d_k =p4d_offset (pgd_k, address); +if (!p4d_present (*p4d_k)) + goto bad_area; + + pud =pud_offset (p4d, address); + pud_k =pud_offset (p4d_k, address);if (!pud_present (*pud_k)) goto bad_area;
arch/arc/mm/highmem.c +3 -1
@@ -111,12 +111,14 @@ EXPORT_SYMBOL(__kunmap_atomic); static noinline pte_t * __init alloc_kmap_pgtable(unsigned long kvaddr) {pgd_t *pgd_k; +p4d_t *p4d_k;pud_t *pud_k;pmd_t *pmd_k;pte_t *pte_k; pgd_k =pgd_offset_k (kvaddr); - pud_k =pud_offset (pgd_k, kvaddr); + p4d_k =p4d_offset (pgd_k, kvaddr); + pud_k =pud_offset (p4d_k, kvaddr); pmd_k =pmd_offset (pud_k, kvaddr); pte_k = (pte_t *)memblock_alloc_low (PAGE_SIZE ,PAGE_SIZE );
Signature · 1 line
2.20.1
Re: [PATCH v2 1/5] ARC: mm: remove __ARCH_USE_5LEVEL_HACK
Quoted 3 lines · Vineet Gupta
This is a non-functional change anyways since ARC has software page walker with 2 lookup levels (pgd -> pte)
Could we encourage other architectures to do the same, and get rid of all uses of __ARCH_USE_5LEVEL_HACK?
Linus
Re: [PATCH v2 1/5] ARC: mm: remove __ARCH_USE_5LEVEL_HACK
Quoted 7 lines · Linus Torvalds
Quoted 2 lines · Vineet Gupta
This is a non-functional change anyways since ARC has software page walker with 2 lookup levels (pgd -> pte)
Could we encourage other architectures to do the same, and get rid of all uses of
__ARCH_USE_5LEVEL_HACK?
IMHO this should have been done at the onset. The actual changes don't seem that difficult, just need to add the missing p4d calls although I can sympathize with hassles of coordinating/building/testing/yadi yada cross arch.
OTOH, the [45]LEVEL_HACK seem like a nice way to "fold" the levels: the skipped/folded level vanishes completely. Among others it does things like
#define p4d_t pgd_t
On ARC 2-level code:
free_pte_range
pmd_pgtable(*pmd); ((((((*pmd).pud).pgd))) & PAGE_MASK)); <-- 5LEVEL_HACK vs. ((((((((*pmd).pud).p4d).pgd)))) & PAGE_MASK )); <-- w/o 5LEVEL_HACK pmd_clear(pmd); *(pmd)).pud).pgd)))) = 0 vs. *(pmd)).pud).p4d).pgd)))) = 0
So we may not be able to fix all he historical misgivigs, but this might alleviate the pain a bit. I'll try to dabble a bit.
Thx for taking a look and te ACKs.
-Vineet
PATCH v2 2/5 asm-generic/tlb: stub out pud_free_tlb() if nopud ...
In reply to Vineet Gupta... independent of __ARCH_HAS_4LEVEL_HACK
This came up when removing __ARCH_HAS_5LEVEL_HACK for ARC as code bloat from this routine which is not required in a 2-level paging setup
Note: The primary change is alternate defines for pud_free_tlb() but while here removed empty stubs for __pud_free_tlb as, which is only called from pud_free_tlb(), being conditionalized already
| bloat-o-meter2 vmlinux-B-elide-ARCH_USE_5LEVEL_HACK vmlinux-C-elide-pud_free_tlb | add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-104 (-104) | function old new delta | free_pgd_range 656 552 -104 | Total: Before=4137276, After=4137172, chg -1.000000%
--- include/asm-generic/4level-fixup.h | 1 - include/asm-generic/pgtable-nopud.h | 2 +- include/asm-generic/tlb.h | 2 -- 3 files changed, 1 insertion(+), 4 deletions(-)
include/asm-generic/4level-fixup.h +0 -1
@@ -30,7 +30,6 @@#undef pud_free_tlb#define pud_free_tlb (tlb, x, addr) do { } while (0)#define pud_free (mm, x) do { } while (0) -#define __pud_free_tlb (tlb, x, addr) do { } while (0)#undef pud_addr_end#define pud_addr_end (addr, end) (end)
include/asm-generic/pgtable-nopud.h +1 -1
@@ -59,7 +59,7 @@ static inline pud_t *pud_offset(p4d_t *p4d, unsigned long address)*/ #define pud_alloc_one (mm, address) NULL#define pud_free (mm, x) do { } while (0) -#define __pud_free_tlb (tlb, x, a) do { } while (0) +#define pud_free_tlb (tlb, x, a) do { } while (0)#undef pud_addr_end#define pud_addr_end (addr, end) (end)
include/asm-generic/tlb.h +0 -2
@@ -584,7 +584,6 @@ static inline void tlb_end_vma(struct mmu_gather *tlb, struct vm_area_struct *vm } while (0)#endif -#ifndef __ARCH_HAS_4LEVEL_HACK#ifndef pud_free_tlb#define pud_free_tlb (tlb, pudp, address) \ do { \
@@ -594,7 +593,6 @@ static inline void tlb_end_vma(struct mmu_gather *tlb, struct vm_area_struct *vm__pud_free_tlb (tlb, pudp, address); \ } while (0) #endif -#endif #ifndef __ARCH_HAS_5LEVEL_HACK#ifndef p4d_free_tlb
Signature · 1 line
2.20.1
Re: [PATCH v2 2/5] asm-generic/tlb: stub out pud_free_tlb() if nopud ...
Quoted 3 lines · Vineet Gupta
This came up when removing
__ARCH_HAS_5LEVEL_HACKfor ARC as code bloat from this routine which is not required in a 2-level paging setup
Ack, looks good.
Linus
PATCH v2 3/5 asm-generic/tlb: stub out p4d_free_tlb() if nop4d ...
In reply to Vineet Gupta... independent of __ARCH_HAS_5LEVEL_HACK
This came up when removing __ARCH_HAS_5LEVEL_HACK for ARC as code bloat from this routine not required in a 2-level paging setup
| bloat-o-meter2 vmlinux-C-elide-pud_free_tlb vmlinux-D-elide-p4d_free_tlb | add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-104 (-104) | function old new delta | free_pgd_range 552 422 -130 | Total: Before=4137172, After=4137042, chg -1.000000%
--- include/asm-generic/5level-fixup.h | 1 - include/asm-generic/pgtable-nop4d.h | 2 +- include/asm-generic/tlb.h | 2 -- 3 files changed, 1 insertion(+), 4 deletions(-)
include/asm-generic/5level-fixup.h +0 -1
@@ -51,7 +51,6 @@ static inline int p4d_present(p4d_t p4d)#undef p4d_free_tlb#define p4d_free_tlb (tlb, x, addr) do { } while (0)#define p4d_free (mm, x) do { } while (0) -#define __p4d_free_tlb (tlb, x, addr) do { } while (0)#undef p4d_addr_end#define p4d_addr_end (addr, end) (end)
include/asm-generic/pgtable-nop4d.h +1 -1
@@ -50,7 +50,7 @@ static inline p4d_t *p4d_offset(pgd_t *pgd, unsigned long address)*/ #define p4d_alloc_one (mm, address) NULL#define p4d_free (mm, x) do { } while (0) -#define __p4d_free_tlb (tlb, x, a) do { } while (0) +#define p4d_free_tlb (tlb, x, a) do { } while (0)#undef p4d_addr_end#define p4d_addr_end (addr, end) (end)
include/asm-generic/tlb.h +0 -2
@@ -594,7 +594,6 @@ static inline void tlb_end_vma(struct mmu_gather *tlb, struct vm_area_struct *vm } while (0)#endif -#ifndef __ARCH_HAS_5LEVEL_HACK#ifndef p4d_free_tlb#define p4d_free_tlb (tlb, pudp, address) \ do { \
@@ -603,7 +602,6 @@ static inline void tlb_end_vma(struct mmu_gather *tlb, struct vm_area_struct *vm__p4d_free_tlb (tlb, pudp, address); \ } while (0)#endif -#endif #endif /* CONFIG_MMU */
Signature · 1 line
2.20.1
Re: [PATCH v2 3/5] asm-generic/tlb: stub out p4d_free_tlb() if nop4d ...
Quoted 3 lines · Vineet Gupta
This came up when removing
__ARCH_HAS_5LEVEL_HACKfor ARC as code bloat from this routine not required in a 2-level paging setup
Similarly acked,
Linus
PATCH v2 4/5 asm-generic/tlb: stub out pmd_free_tlb() if nopmd
In reply to Vineet GuptaNote that pmd routine folding can be tricky as even in 2-level setup (where pmd is folded) most pmd routines refer to upper levels. This one can surely be elided however.
| bloat-o-meter2 vmlinux-E-elide-p?d_clear_bad vmlinux-F-elide-pmd_free_tlb | add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-112 (-112) | function old new delta | free_pgd_range 422 310 -112 | Total: Before=4137042, After=4136930, chg -1.000000%
--- include/asm-generic/pgtable-nopmd.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/asm-generic/pgtable-nopmd.h +1 -1
@@ -60,7 +60,7 @@ static inline pmd_t * pmd_offset(pud_t * pud, unsigned long address)static inline void pmd_free (struct mm_struct *mm,pmd_t *pmd) { } -#define __pmd_free_tlb (tlb, x, a) do { } while (0) +#define pmd_free_tlb (tlb, x, a) do { } while (0)#undef pmd_addr_end#define pmd_addr_end (addr, end) (end)
Signature · 1 line
2.20.1
PATCH v2 5/5 asm-generic/mm: stub out p{4,d}d_clear_bad() if __PAGETABLE_P{4,u}D_FOLDED
In reply to Vineet GuptaThis removes the code for 2 level paging as seen on ARC
| bloat-o-meter2 vmlinux-D-elide-p4d_free_tlb vmlinux-E-elide-p?d_clear_bad | add/remove: 0/2 grow/shrink: 0/0 up/down: 0/-40 (-40) | function old new delta | pud_clear_bad 20 - -20 | p4d_clear_bad 20 - -20 | Total: Before=4136930, After=4136890, chg -1.000000%
--- include/asm-generic/pgtable.h | 11 +++++++++++ mm/pgtable-generic.c | 8 ++++++++ 2 files changed, 19 insertions(+)
include/asm-generic/pgtable.h +11 -0
@@ -558,8 +558,19 @@ static inline pgprot_t pgprot_modify(pgprot_t oldprot, pgprot_t newprot)* Do the tests inline, but report and clear the bad entry in mm/memory.c. */ void pgd_clear_bad (pgd_t *); + +#ifndef __PAGETABLE_P4D_FOLDEDvoid p4d_clear_bad (p4d_t *); +#else +#define p4d_clear_bad (p4d) do { } while (0) +#endif + +#ifndef __PAGETABLE_PUD_FOLDEDvoid pud_clear_bad (pud_t *); +#else +#define pud_clear_bad (p4d) do { } while (0) +#endif +void pmd_clear_bad (pmd_t *);static inline int pgd_none_or_clear_bad (pgd_t *pgd)
mm/pgtable-generic.c +8 -0
@@ -24,18 +24,26 @@ void pgd_clear_bad(pgd_t *pgd)pgd_clear (pgd); } +#ifndef __PAGETABLE_P4D_FOLDEDvoid p4d_clear_bad (p4d_t *p4d) {p4d_ERROR (*p4d);p4d_clear (p4d); } +#endif +#ifndef __PAGETABLE_PUD_FOLDEDvoid pud_clear_bad (pud_t *pud) {pud_ERROR (*pud);pud_clear (pud); } +#endif +/* +* Note that below can't be stubed out for nopmd case: +* pmd folding is special and typically pmd_* macros refet to upper level +*/ void pmd_clear_bad (pmd_t *pmd) {pmd_ERROR (*pmd);
Signature · 1 line
2.20.1
Re: [PATCH v2 5/5] asm-generic/mm: stub out p{4,d}d_clear_bad() if __PAGETABLE_P{4,u}D_FOLDED
There is couple of typos in the subject of the patch. It has to be
[PATCH v2 5/5] asm-generic/mm: stub out p{4,u}d_clear_bad() if __PAGETABLE_P{4,U}D_FOLDEDOtherwise the patchset looks good to me. You can use my ACK for all patches.
Signature · 1 line
Kirill A. Shutemov