Skip to contentSkip to conversation

Page tables on machines with >2GB RAM

DISCUSSION Matthew Wilcox 3 participants 6 messages Sep 29, 2020 – Oct 4, 2020

lore
Share & tools
lore
#0 lore raw

I think we can end up truncating a PMD or PGD entry (I get confused easily about levels of the page tables; bear with me)

/* NOTE: even on 64 bits, these entries are __u32 because we allocate
* the pmd and pgd in ZONE_DMA (i.e. under 4GB) */
typedef struct { __u32 pgd; } pgd_t;
...
typedef struct { __u32 pmd; } pmd_t;

...

        pgd_t *pgd = (pgd_t *)__get_free_pages(GFP_KERNEL,
                                               PGD_ALLOC_ORDER);
...
        return (pmd_t *)__get_free_pages(GFP_PGTABLE_KERNEL, PMD_ORDER);

so if we have more than 2GB of RAM, we can allocate a page with the top bit set, which we interpret to mean PAGE_PRESENT in the TLB miss handler and mask it off, causing us to load the wrong page for the next level of the page table walk.

Have I missed something?

Oh and I think this bug was introduced in 2004 ...

#1 lore raw
Quoted 16 lines · Matthew Wilcox

I think we can end up truncating a PMD or PGD entry (I get confused easily about levels of the page tables; bear with me)

/* NOTE: even on 64 bits, these entries are __u32 because we allocate
* the pmd and pgd in ZONE_DMA (i.e. under 4GB) */
typedef struct { __u32 pgd; } pgd_t;
...
typedef struct { __u32 pmd; } pmd_t;

...

        pgd_t *pgd = (pgd_t *)__get_free_pages(GFP_KERNEL,
                                               PGD_ALLOC_ORDER);
...
        return (pmd_t *)__get_free_pages(GFP_PGTABLE_KERNEL, PMD_ORDER);
Quoted 6 lines

so if we have more than 2GB of RAM, we can allocate a page with the top bit set, which we interpret to mean PAGE_PRESENT in the TLB miss handler and mask it off, causing us to load the wrong page for the next level of the page table walk.

Have I missed something?

Yes, yes I have.

We store the PFN, not the physical address. So we have 28 bits for storing the PFN and 4 bits for the PxD bits, supporting 28 + 12 = 40 bits (1TB) of physical address space.

#2 lore raw
Quoted 22 lines · Matthew Wilcox
Quoted 21 lines · Matthew Wilcox

I think we can end up truncating a PMD or PGD entry (I get confused easily about levels of the page tables; bear with me)

/* NOTE: even on 64 bits, these entries are __u32 because we allocate
* the pmd and pgd in ZONE_DMA (i.e. under 4GB) */
typedef struct { __u32 pgd; } pgd_t;
...
typedef struct { __u32 pmd; } pmd_t;

...

        pgd_t *pgd = (pgd_t *)__get_free_pages(GFP_KERNEL,
                                               PGD_ALLOC_ORDER);
...
        return (pmd_t *)__get_free_pages(GFP_PGTABLE_KERNEL, PMD_ORDER);

so if we have more than 2GB of RAM, we can allocate a page with the top bit set, which we interpret to mean PAGE_PRESENT in the TLB miss handler and mask it off, causing us to load the wrong page for the next level of the page table walk.

Quoted 6 lines
Quoted 1 line

Have I missed something?

Yes, yes I have.

We store the PFN, not the physical address. So we have 28 bits for storing the PFN and 4 bits for the PxD bits, supporting 28 + 12 = 40 bits (1TB) of physical address space.

The comment in pgalloc.h says 8TB?  I think improving the description as to how this works would be welcome.

Signature · 1 line
John David Anglin  [email protected]
#3 lore raw
Quoted 23 lines · John David Anglin
Quoted 22 lines · Matthew Wilcox
Quoted 21 lines · Matthew Wilcox

I think we can end up truncating a PMD or PGD entry (I get confused easily about levels of the page tables; bear with me)

/* NOTE: even on 64 bits, these entries are __u32 because we allocate
* the pmd and pgd in ZONE_DMA (i.e. under 4GB) */
typedef struct { __u32 pgd; } pgd_t;
...
typedef struct { __u32 pmd; } pmd_t;

...

        pgd_t *pgd = (pgd_t *)__get_free_pages(GFP_KERNEL,
                                               PGD_ALLOC_ORDER);
...
        return (pmd_t *)__get_free_pages(GFP_PGTABLE_KERNEL, PMD_ORDER);

so if we have more than 2GB of RAM, we can allocate a page with the top bit set, which we interpret to mean PAGE_PRESENT in the TLB miss handler and mask it off, causing us to load the wrong page for the next level of the page table walk.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Have I missed something?

Yes, yes I have.

We store the PFN, not the physical address. So we have 28 bits for storing the PFN and 4 bits for the PxD bits, supporting 28 + 12 = 40 bits (1TB) of physical address space.

The comment in pgalloc.h says 8TB?  I think improving the description as to how this works would be welcome.

It's talking about 8TB of virtual address space. But I think it's wrong. On 64-bit,

Each PTE defines a 4kB region of address space (ie one page). Each PMD is a 4kB allocation with 8-byte entries, so covers 512 * 4kB = 2MB Each PGD is an 8kB allocation with 4-byte entries, so covers 2048 * 2M = 4GB The top-level allocation is a 32kB allocation, but the first 8kB is used for the first PGD, so it covers 24kB / 4 bytes * 4GB = 24TB.

I think the top level allocation was supposed to be an order-2 allocation, which would be an 8TB address space, but it's order-3.

There's a lot of commentary which disagrees with the code. For example,

#define PMD_ORDER 1 /* Number of pages per pmd */

That's just not true; an order-1 allocation is 2 pages, not 1.

#4 lore raw
Quoted 30 lines · Matthew Wilcox
Quoted 29 lines · John David Anglin
Quoted 28 lines · Matthew Wilcox
Quoted 16 lines · Matthew Wilcox

I think we can end up truncating a PMD or PGD entry (I get confused easily about levels of the page tables; bear with me)

/* NOTE: even on 64 bits, these entries are __u32 because we allocate
* the pmd and pgd in ZONE_DMA (i.e. under 4GB) */
typedef struct { __u32 pgd; } pgd_t;
...
typedef struct { __u32 pmd; } pmd_t;

...

        pgd_t *pgd = (pgd_t *)__get_free_pages(GFP_KERNEL,
                                               PGD_ALLOC_ORDER);
...
        return (pmd_t *)__get_free_pages(GFP_PGTABLE_KERNEL, PMD_ORDER);
Quoted 6 lines

so if we have more than 2GB of RAM, we can allocate a page with the top bit set, which we interpret to mean PAGE_PRESENT in the TLB miss handler and mask it off, causing us to load the wrong page for the next level of the page table walk.

Have I missed something?

Yes, yes I have.

We store the PFN, not the physical address. So we have 28 bits for storing the PFN and 4 bits for the PxD bits, supporting 28 + 12 = 40 bits (1TB) of physical address space.

Quoted 8 lines
Quoted 2 lines

The comment in pgalloc.h says 8TB?  I think improving the description as to how this works would be welcome.

It's talking about 8TB of virtual address space. But I think it's wrong. On 64-bit,

Each PTE defines a 4kB region of address space (ie one page). Each PMD is a 4kB allocation with 8-byte entries, so covers 512 * 4kB = 2MB

No, PMD is 4kb allocation with 4-byte entries, so covers 1024 * 4kb = 4MB We always us 4-byte entries, for 32- and 64-bit kernels.

Quoted 1 line

Each PGD is an 8kB allocation with 4-byte entries, so covers 2048 * 2M = 4GB

No. each PGD is a 4kb allocation with 4-byte entries. so covers 1024 * 4MB = 4GB Still, my calculation ends up with 4GB, like yours.

Quoted 2 lines

The top-level allocation is a 32kB allocation, but the first 8kB is used for the first PGD, so it covers 24kB / 4 bytes * 4GB = 24TB.

size of PGD (swapper_pg_dir) is 8k, so we have 8k / 4 bytes * 4GB = 8 TB virtual address space.

At boot we want to map (1 << KERNEL_INITIAL_ORDER) pages (=64MB on 64bit kernel) and for this pmd0 gets pre-allocated with 8k size, and pg0 with 132k to simplify the filling the initial page tables - but that's not relevant for the calculations above.

Quoted 7 lines

I think the top level allocation was supposed to be an order-2 allocation, which would be an 8TB address space, but it's order-3.

There's a lot of commentary which disagrees with the code. For example,

#define PMD_ORDER 1 /* Number of pages per pmd */
That's just not true; an order-1 allocation is 2 pages, not 1.

Yes, that should be fixed up.

Helge

#5 lore raw
Quoted 9 lines · Helge Deller
Quoted 5 lines · Matthew Wilcox

It's talking about 8TB of virtual address space. But I think it's wrong. On 64-bit,

Each PTE defines a 4kB region of address space (ie one page). Each PMD is a 4kB allocation with 8-byte entries, so covers 512 * 4kB = 2MB

No, PMD is 4kb allocation with 4-byte entries, so covers 1024 * 4kb = 4MB We always us 4-byte entries, for 32- and 64-bit kernels.

#if CONFIG_PGTABLE_LEVELS == 3
#define PGD_ORDER 1 /* Number of pages per pgd */
#define PMD_ORDER 1 /* Number of pages per pmd */
#define PGD_ALLOC_ORDER (2 + 1) /* first pgd contains pmd */
...
#if CONFIG_PGTABLE_LEVELS == 3
...
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)

{
        return (pmd_t *)__get_free_pages(GFP_PGTABLE_KERNEL, PMD_ORDER);
}

We're definitely doing an 8kB allocation. If we should be allocating 4kB, then PMD_ORDER should be 0.

The 32-bit entries, even on 64-bit are a nice hack. I think that just means we're over-allocating memory for the page tables.

Quoted 4 lines
Quoted 1 line

Each PGD is an 8kB allocation with 4-byte entries, so covers 2048 * 2M = 4GB

No. each PGD is a 4kb allocation with 4-byte entries. so covers 1024 * 4MB = 4GB Still, my calculation ends up with 4GB, like yours.

Again, I think there's an order vs count confusion here.

Quoted 10 lines
Quoted 2 lines

The top-level allocation is a 32kB allocation, but the first 8kB is used for the first PGD, so it covers 24kB / 4 bytes * 4GB = 24TB.

size of PGD (swapper_pg_dir) is 8k, so we have 8k / 4 bytes * 4GB = 8 TB virtual address space.

At boot we want to map (1 << KERNEL_INITIAL_ORDER) pages (=64MB on 64bit kernel) and for this pmd0 gets pre-allocated with 8k size, and pg0 with 132k to simplify the filling the initial page tables - but that's not relevant for the calculations above.

I was talking about pgd_alloc():

        pgd_t *pgd = (pgd_t *)__get_free_pages(GFP_KERNEL,
                                               PGD_ALLOC_ORDER);

where we allocate 8 * 4kB pages

Quoted 11 lines
Quoted 7 lines

I think the top level allocation was supposed to be an order-2 allocation, which would be an 8TB address space, but it's order-3.

There's a lot of commentary which disagrees with the code. For example,

#define PMD_ORDER 1 /* Number of pages per pmd */
That's just not true; an order-1 allocation is 2 pages, not 1.

Yes, that should be fixed up.

Helge