X86-32: Allocate 256 bytes for pgd in PAE paging
PATCH v2 1/1 X86-32: Allocate 256 bytes for pgd in PAE paging
From: Fenghua Yu <[email protected]>
X86 32-bit machine and kernel use PAE paging, which currently wastes about 4K of memory per process on Linux where we have to reserve an entire page to support a single 256-byte PGD structure. It would be a very good thing if we could eliminate that wastage.
--- arch/x86/mm/pgtable.c | 42 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 3 deletions(-)
arch/x86/mm/pgtable.c +39 -3
@@ -1,5 +1,6 @@#include <linux/mm.h> #include <linux/gfp.h> +#include <linux/slab.h> #include <asm/pgalloc.h> #include <asm/pgtable.h> #include <asm/tlb.h>
@@ -271,12 +272,46 @@ static void pgd_prepopulate_pmd(struct mm_struct *mm, pgd_t *pgd, pmd_t *pmds[]) } } +/* +* Xen paravirt assumes pgd table should be in one page. pgd in 64 bit also +* needs to be in one page. +* +* But PAE without Xen only needs to allocate 256 bytes for pgd. +* +* So if kernel is compiled as PAE model without Xen, we allocate 256 bytes +* for pgd entries to save memory space. +* +* In other cases, one page is allocated for pgd. In theory, a kernel +* in PAE mode not running in Xen could allocate 256 bytes for pgd +* as well. But that will make the allocation and free more complex +* but not useful in reality. To simplify the code and testing, we just +* allocate one page when CONFIG_XEN is enabled regardelss kernel is running +* in Xen or not. +*/ +static inline pgd_t *_pgd_alloc (void ) +{ +#if defined(CONFIG_X86_PAE ) && !defined(CONFIG_XEN ) +return kmalloc (sizeof (pgdval_t) *PTRS_PER_PGD ,PGALLOC_GFP ); +#else +return (pgd_t *)__get_free_page (PGALLOC_GFP ); +#endif +} + +static inline void _pgd_free (pgd_t *pgd) +{ +#if defined(CONFIG_X86_PAE ) && !defined(CONFIG_XEN ) +kfree (pgd); +#else +free_page ((unsigned long )pgd); +#endif +} +pgd_t *pgd_alloc (struct mm_struct *mm) {pgd_t *pgd;pmd_t *pmds[PREALLOCATED_PMDS ]; - pgd = (pgd_t *)__get_free_page (PGALLOC_GFP ); + pgd =_pgd_alloc ();if (pgd ==NULL ) goto out;
@@ -306,7 +341,7 @@ pgd_t *pgd_alloc(struct mm_struct *mm) out_free_pmds:free_pmds (pmds); out_free_pgd: -free_page ((unsigned long )pgd); +_pgd_free (pgd); out:return NULL ; }
@@ -316,7 +351,8 @@ void pgd_free(struct mm_struct *mm, pgd_t *pgd)pgd_mop_up_pmds (mm, pgd);pgd_dtor (pgd);paravirt_pgd_free (mm, pgd); -free_page ((unsigned long )pgd); +_pgd_free (pgd); + }/*
Signature · 1 line
1.8.1.2
arch/x86/mm/pgtable.c static void pgd_prepopulate_pmd(struct mm_struct *mm, pgd_t *pgd, pmd_t *pmds[]) · line 298
Quoted 8 lines · Fenghua Yu
arch/x86/mm/pgtable.c +8
-0+static inline pgd_t *_pgd_alloc (void ) +{ +#if defined(CONFIG_X86_PAE ) && !defined(CONFIG_XEN ) +return kmalloc (sizeof (pgdval_t) *PTRS_PER_PGD ,PGALLOC_GFP ); +#else +return (pgd_t *)__get_free_page (PGALLOC_GFP ); +#endif +}
I'm looking at:
"Figure 4-7. Formats of CR3 and Paging-Structure Entries with
PAE Paging"
in the SDM. It makes it pretty clear that the lower 5 bits of cr3 are ignored in PAE mode. That means we have to be 32-byte (or greater) aligned, right? Does kmalloc() guarantee that?
IOW, do *ALL* of the sl*b allocators in all of their forms with all of their debugging options guarantee 32-byte alignment when allocating 256-byte objects?
I know we at least try to align to a cacheline, which would be good enough, but I'm fuzzy on what we *guarantee*.
Quoted 3 lines · Dave Hansen
IOW, do *ALL* of the sl*b allocators in all of their forms with all of their debugging options guarantee 32-byte alignment when allocating 256-byte objects?
No. For that the arch has to set a macro call ARCH_KMALLOC_MINALIGN or ARCH_DMA_MINALIGN. Default alignment is to a word boundary.
Quoted 2 lines
I know we at least try to align to a cacheline, which would be good enough, but I'm fuzzy on what we *guarantee*.
Sorry we only do this if requested for a slab cache.
Quoted 6 lines · Christoph Lameter
Quoted 3 lines · Dave Hansen
Quoted 3 lines
IOW, do *ALL* of the sl*b allocators in all of their forms with all of their debugging options guarantee 32-byte alignment when allocating 256-byte objects?
No. For that the arch has to set a macro call
ARCH_KMALLOC_MINALIGNorARCH_DMA_MINALIGN. Default alignment is to a word boundary.
OK, sounds like this is going to need its own slab. It will make the patch a bit bigger but shouldn't be all that much more complicated.
H. Peter Anvin
reviewerQuoted 7 lines · Fenghua Yu
From: Fenghua Yu <[email protected]>
X86 32-bit machine and kernel use PAE paging, which currently wastes about 4K of memory per process on Linux where we have to reserve an entire page to support a single 256-byte PGD structure. It would be a very good thing if we could eliminate that wastage.
4*8 = 32 bytes, where did 256 bytes come from?
-hpa
Quoted 9 lines
Sent: Thursday, December 18, 2014 9:26 AM
To: Yu, Fenghua; Thomas Gleixner; H. Peter Anvin; Ingo Molnar; Williamson, Glenn P
Cc: linux-kernel; x86
Subject: Re: [PATCH v2] X86-32: Allocate 256 bytes for pgd in PAE pagingQuoted 2 lines · Fenghua Yu
From: Fenghua Yu <[email protected]>
Quoted 7 lines
Quoted 5 lines
X86 32-bit machine and kernel use PAE paging, which currently wastes about 4K of memory per process on Linux where we have to reserve an entire page to support a single 256-byte PGD structure. It would be a very good thing if we could eliminate that wastage.
4*8 = 32 bytes, where did 256 bytes come from?
You are right. It should be 32 bytes. I will change the wording in future patch. The real calculation in code is right though.
Thanks.
-Fenghua
H. Peter Anvin
reviewerQuoted 3 lines
Quoted 1 line
4*8 = 32 bytes, where did 256 bytes come from?
You are right. It should be 32 bytes. I will change the wording in future patch. The real calculation in code is right though.
I don't know if it makes sense to round up to a cache line. My suspicion is that it won't matter, as these fields will be read-mostly, on the other hand, wasting 32 bytes isn't exactly a problem, either.
-hpa