s390/mm: Add missing ctor/dtor on page table upgrade
Subsystems (3)
Alexander Gordeev
maintainerPATCH 1/1 s390/mm: Add missing ctor/dtor on page table upgrade
Commit 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table") misses the call to pagetable_p4d_ctor() against a newly allocated P4D table in crst_table_upgrade();
Commit 68c601de75d8 ("mm: introduce ctor/dtor at PGD level") misses the call to pagetable_pgd_ctor() against a newly allocated PGD and the call to pagetable_dtor() against a newly allocated P4D that is about to be freed on crst_table_upgrade() PGD upgrade fail path.
The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.
---
The patch is against:
git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git next-20250123
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm mm-stable
--- arch/s390/mm/pgalloc.c | 3 +++ 1 file changed, 3 insertions(+)
arch/s390/mm/pgalloc.c +3 -0
@@ -88,12 +88,14 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)if (unlikely (!p4d)) goto err_p4d;crst_table_init (p4d, _REGION2_ENTRY_EMPTY); +pagetable_p4d_ctor (virt_to_ptdesc (p4d)); }if (end > _REGION1_SIZE) { pgd =crst_table_alloc (mm);if (unlikely (!pgd)) goto err_pgd;crst_table_init (pgd, _REGION1_ENTRY_EMPTY); +pagetable_pgd_ctor (virt_to_ptdesc (pgd)); }spin_lock_bh (&mm->page_table_lock);
@@ -130,6 +132,7 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)return 0 ; err_pgd: +pagetable_dtor (virt_to_ptdesc (p4d));crst_table_free (mm, p4d); err_p4d:return -ENOMEM ;
Signature · 1 line
2.45.2
Quoted 47 lines · On Thu, 23 Jan 2025 17:03:49 +0100 Alexander Gordeev <[email protected]>…
Commit
78966b550289("s390: pgtable: add statistics for PUD and P4D level page table") misses the call topagetable_p4d_ctor()against a newly allocated P4D table incrst_table_upgrade();Commit
68c601de75d8("mm: introduce ctor/dtor at PGD level") misses the call topagetable_pgd_ctor()against a newly allocated PGD and the call topagetable_dtor()against a newly allocated P4D that is about to be freed oncrst_table_upgrade()PGD upgrade fail path.The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.
Reported-by Heiko Carstens <[email protected]>Suggested-by Heiko Carstens <[email protected]>Fixes 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table")Fixes 68c601de75d8 ("mm: introduce ctor/dtor at PGD level")Signed-off-by Alexander Gordeev <[email protected]>---
The patch is against:
git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.gitnext-20250123
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mmmm-stable--- arch/s390/mm/pgalloc.c | 3 +++ 1 file changed, 3 insertions(+)arch/s390/mm/pgalloc.c +2
-0@@ -88,12 +88,14 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)if (unlikely (!p4d)) goto err_p4d;crst_table_init (p4d, _REGION2_ENTRY_EMPTY); +pagetable_p4d_ctor (virt_to_ptdesc (p4d)); }if (end > _REGION1_SIZE) { pgd =crst_table_alloc (mm);if (unlikely (!pgd)) goto err_pgd;crst_table_init (pgd, _REGION1_ENTRY_EMPTY); +pagetable_pgd_ctor (virt_to_ptdesc (pgd)); }spin_lock_bh (&mm->page_table_lock);
arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 138
Quoted 7 lines
arch/s390/mm/pgalloc.c +1
-0@@ -130,6 +132,7 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)return 0 ; err_pgd: +pagetable_dtor (virt_to_ptdesc (p4d));crst_table_free (mm, p4d); err_p4d:return -ENOMEM ;
Thanks, looks good, and nice catch.
Quoted 18 lines · Alexander Gordeev
Commit
78966b550289("s390: pgtable: add statistics for PUD and P4D level page table") misses the call topagetable_p4d_ctor()against a newly allocated P4D table incrst_table_upgrade();Commit
68c601de75d8("mm: introduce ctor/dtor at PGD level") misses the call topagetable_pgd_ctor()against a newly allocated PGD and the call topagetable_dtor()against a newly allocated P4D that is about to be freed oncrst_table_upgrade()PGD upgrade fail path.The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.
Reported-by Heiko Carstens <[email protected]>Suggested-by Heiko Carstens <[email protected]>Fixes 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table")Fixes 68c601de75d8 ("mm: introduce ctor/dtor at PGD level")Signed-off-by Alexander Gordeev <[email protected]>
Quoted 9 lines
---
The patch is against:git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git next-20250123 git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm mm-stable --- arch/s390/mm/pgalloc.c | 3 +++ 1 file changed, 3 insertions(+)
Thanks!
Quoted 18 lines · Alexander Gordeev
Commit
78966b550289("s390: pgtable: add statistics for PUD and P4D level page table") misses the call topagetable_p4d_ctor()against a newly allocated P4D table incrst_table_upgrade();Commit
68c601de75d8("mm: introduce ctor/dtor at PGD level") misses the call topagetable_pgd_ctor()against a newly allocated PGD and the call topagetable_dtor()against a newly allocated P4D that is about to be freed oncrst_table_upgrade()PGD upgrade fail path.The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.
Reported-by Heiko Carstens <[email protected]>Suggested-by Heiko Carstens <[email protected]>Fixes 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table")Fixes 68c601de75d8 ("mm: introduce ctor/dtor at PGD level")Signed-off-by Alexander Gordeev <[email protected]>
Quoted 5 lines
---
The patch is against:
git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.gitnext-20250123
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mmmm-stable
Thank you for putting together this patch! I was completely unaware of this "upgrade" path on s390.
arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 91
Quoted 13 lines
--- arch/s390/mm/pgalloc.c | 3 +++ 1 file changed, 3 insertions(+)arch/s390/mm/pgalloc.c +1
-0@@ -88,12 +88,14 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)if (unlikely (!p4d)) goto err_p4d;crst_table_init (p4d, _REGION2_ENTRY_EMPTY); +pagetable_p4d_ctor (virt_to_ptdesc (p4d));
This block seems to be equivalent to p4d_alloc_one(), maybe it would be preferable to just call p4d_alloc_one() here to avoid further mismatches in the future (and reduce duplication)?
arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 98
Quoted 7 lines
arch/s390/mm/pgalloc.c +1
-0}if (end > _REGION1_SIZE) { pgd =crst_table_alloc (mm);if (unlikely (!pgd)) goto err_pgd;crst_table_init (pgd, _REGION1_ENTRY_EMPTY); +pagetable_pgd_ctor (virt_to_ptdesc (pgd));
I was hoping this might be equivalent to pgd_alloc() but it does not include a call to crst_table_init(). Since adding it would be apparently undesirable (having read the other thread), it seems reasonable to add the explicit constructor call.
arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 136
Quoted 9 lines
arch/s390/mm/pgalloc.c +1
-0}spin_lock_bh (&mm->page_table_lock);@@ -130,6 +132,7 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)return 0 ; err_pgd: +pagetable_dtor (virt_to_ptdesc (p4d));crst_table_free (mm, p4d);
Similarly, this could be a call to p4d_free().
- Kevin
Quoted 2 lines
arch/s390/mm/pgalloc.c +0
-0err_p4d:return -ENOMEM ;
Alexander Gordeev
maintainerOn Fri, Jan 24, 2025 at 08:58:07AM +0100, Kevin Brodsky wrote:
Kevin,
...
Quoted 2 lines
Thank you for putting together this patch! I was completely unaware of this "upgrade" path on s390.
...
Quoted 5 lines
Quoted 5 lines
arch/s390/mm/pgalloc.c +0
-0@@ -88,12 +88,14 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)
Quoted 8 lines
arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 91
Quoted 4 lines
arch/s390/mm/pgalloc.c +1
-0if (unlikely (!p4d)) goto err_p4d;crst_table_init (p4d, _REGION2_ENTRY_EMPTY); +pagetable_p4d_ctor (virt_to_ptdesc (p4d));This block seems to be equivalent to
p4d_alloc_one(), maybe it would be preferable to just callp4d_alloc_one()here to avoid further mismatches in the future (and reduce duplication)?
Yes, I wanted to do exactly that (together with p4d_free() you noticed), but then the whole thing with open coded pagetable_pgd_ctor() below gets inconsistent. I would prefer to keep it this way as of now.
Quoted 4 lines
Quoted 4 lines
arch/s390/mm/pgalloc.c +0
-0}if (end > _REGION1_SIZE) { pgd =crst_table_alloc (mm);if (unlikely (!pgd))
Quoted 8 lines
arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 98
Quoted 3 lines
arch/s390/mm/pgalloc.c +1
-0goto err_pgd;crst_table_init (pgd, _REGION1_ENTRY_EMPTY); +pagetable_pgd_ctor (virt_to_ptdesc (pgd));I was hoping this might be equivalent to
pgd_alloc()but it does not include a call tocrst_table_init(). Since adding it would be apparently undesirable (having read the other thread), it seems reasonable to add the explicit constructor call.
We were thinking about a follow-up cleanup that addresses it all, but this patch is a targeted fix to catch up your and Qi Zheng series in the still open merge window.
Quoted 6 lines
Quoted 6 lines
arch/s390/mm/pgalloc.c +0
-0}spin_lock_bh (&mm->page_table_lock);@@ -130,6 +132,7 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)return 0 ;
Quoted 10 lines
arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 136
Quoted 3 lines
arch/s390/mm/pgalloc.c +1
-0err_pgd: +pagetable_dtor (virt_to_ptdesc (p4d));crst_table_free (mm, p4d);Similarly, this could be a call to
p4d_free().- Kevin
Quoted 2 lines
arch/s390/mm/pgalloc.c +0
-0err_p4d:return -ENOMEM ;
Thanks for the review!
Kevin Brodsky
Reviewed-byQuoted 11 lines · Alexander Gordeev
Quoted 11 lines
arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 98
Quoted 7 lines
arch/s390/mm/pgalloc.c +1
-0}if (end > _REGION1_SIZE) { pgd =crst_table_alloc (mm);if (unlikely (!pgd)) goto err_pgd;crst_table_init (pgd, _REGION1_ENTRY_EMPTY); +pagetable_pgd_ctor (virt_to_ptdesc (pgd));I was hoping this might be equivalent to
pgd_alloc()but it does not include a call tocrst_table_init(). Since adding it would be apparently undesirable (having read the other thread), it seems reasonable to add the explicit constructor call.
Quoted 3 lines
We were thinking about a follow-up cleanup that addresses it all, but this patch is a targeted fix to catch up your and Qi Zheng series in the still open merge window.
Sure that's fair, this patch does fix the issue and the cleanup can wait until the next cycle.
- Kevin
Heiko Carstens
maintainerQuoted 22 lines · Kevin Brodsky
Quoted 21 lines · Alexander Gordeev
Commit
78966b550289("s390: pgtable: add statistics for PUD and P4D level page table") misses the call topagetable_p4d_ctor()against a newly allocated P4D table incrst_table_upgrade();Commit
68c601de75d8("mm: introduce ctor/dtor at PGD level") misses the call topagetable_pgd_ctor()against a newly allocated PGD and the call topagetable_dtor()against a newly allocated P4D that is about to be freed oncrst_table_upgrade()PGD upgrade fail path.The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.
Reported-by Heiko Carstens <[email protected]>Suggested-by Heiko Carstens <[email protected]>Fixes 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table")Fixes 68c601de75d8 ("mm: introduce ctor/dtor at PGD level")Signed-off-by Alexander Gordeev <[email protected]>---
The patch is against:
Quoted 5 lines
Quoted 2 lines
git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.gitnext-20250123
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mmmm-stableThank you for putting together this patch! I was completely unaware of this "upgrade" path on s390.
This whole thing is even worse than it looks after this patch. With page table upgrade taken into account we still have the oddity that e.g. a previous pgd becomes a pud or p4d, which means that ctor and dtor functions might be called for different levels for the same page table. As of now this is ok, since they do all the same.
As a quick fix this patch is ok, and most likely it will be ok for a long time, but it doesn't give me good feeling :)
In addition, looking at [1] where page table accounting was introduced: it is really meant to reflect the memory consumption used by page tables. This might work for nearly all architectures which have the same page table size for every level; but on s390 the lowest level comes with 4kb page tables while all other levels come with 16kb page tables.
Therefore at least on s390 you really can't tell how much memory is consumed by page tables by only looking at nr_page_table_pages. It _might_ make sense to introduce a factor of four for page table accounting for higher levels, so those numbers make at least some sense; but not sure about that.
[1] https://lore.kernel.org/all/[email protected]/T/#mb6c6f2a84ded27cd9b3d54140dde1d5a75c74735
Heiko Carstens
maintainerQuoted 7 lines · Heiko Carstens
On Fri, Jan 24, 2025 at 08:58:07AM +0100, Kevin Brodsky wrote:
In addition, looking at [1] where page table accounting was introduced: it is really meant to reflect the memory consumption used by page tables. This might work for nearly all architectures which have the same page table size for every level; but on s390 the lowest level comes with 4kb page tables while all other levels come with 16kb page tables.
Quoted 5 lines
Therefore at least on s390 you really can't tell how much memory is consumed by page tables by only looking at
nr_page_table_pages. It _might_ make sense to introduce a factor of four for page table accounting for higher levels, so those numbers make at least some sense; but not sure about that.
Ah, this is actually not true at all, since we have
static inline void __lruvec_stat_add_folio(struct folio *folio,
enum node_stat_item idx)
{
__lruvec_stat_mod_folio(folio, idx, folio_nr_pages(folio));
}which will do exactly what we want. So this part is not a problem.
Quoted 27 lines · Heiko Carstens
Quoted 22 lines · Kevin Brodsky
Quoted 21 lines · Alexander Gordeev
Commit
78966b550289("s390: pgtable: add statistics for PUD and P4D level page table") misses the call topagetable_p4d_ctor()against a newly allocated P4D table incrst_table_upgrade();Commit
68c601de75d8("mm: introduce ctor/dtor at PGD level") misses the call topagetable_pgd_ctor()against a newly allocated PGD and the call topagetable_dtor()against a newly allocated P4D that is about to be freed oncrst_table_upgrade()PGD upgrade fail path.The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.
Reported-by Heiko Carstens <[email protected]>Suggested-by Heiko Carstens <[email protected]>Fixes 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table")Fixes 68c601de75d8 ("mm: introduce ctor/dtor at PGD level")Signed-off-by Alexander Gordeev <[email protected]>---
The patch is against:Quoted 4 lines
Quoted 2 lines
git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.gitnext-20250123
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mmmm-stableThank you for putting together this patch! I was completely unaware of this "upgrade" path on s390.
Quoted 8 lines
This whole thing is even worse than it looks after this patch. With page table upgrade taken into account we still have the oddity that e.g. a previous pgd becomes a pud or p4d, which means that ctor and dtor functions might be called for different levels for the same page table. As of now this is ok, since they do all the same.
As a quick fix this patch is ok, and most likely it will be ok for a long time, but it doesn't give me good feeling :)
This could clearly cause trouble if the pud/p4d/pgd constructors start to diverge. In principle this could be addressed when upgrading by calling the dtor on the old pgd and then the ctor on the new pud/p4d. That said since there is now only one dtor, and the ctor are all the same, it's probably good enough.
I'll mention in passing that my own series (that introduced PGD-level ctor/dtor) really aimed at supporting the kpkeys page table protection RFC [1], where hooks are inserted into the ctor/dtor to modify the pkey page tables are mapped with. No idea if that would work with this sort of upgrade path, but since s390 does not support pkeys this is just a theoretical issue :)
- Kevin
[1]
https://lore.kernel.org/linux-hardening/[email protected]/