Skip to contentSkip to conversation

s390/mm: Add missing ctor/dtor on page table upgrade

PATCH Applied Alexander Gordeev 5 participants 9 messages Jan 23, 2025 – Jan 24, 2025

Subsystems (3)
lore
Share & tools
lore

Alexander Gordeev

author maintainer
#0 lore raw

PATCH 1/1 s390/mm: Add missing ctor/dtor on page table upgrade

Commit 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table") misses the call to pagetable_p4d_ctor() against a newly allocated P4D table in crst_table_upgrade();

Commit 68c601de75d8 ("mm: introduce ctor/dtor at PGD level") misses the call to pagetable_pgd_ctor() against a newly allocated PGD and the call to pagetable_dtor() against a newly allocated P4D that is about to be freed on crst_table_upgrade() PGD upgrade fail path.

The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.

Fixes 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table")
Fixes 68c601de75d8 ("mm: introduce ctor/dtor at PGD level")

---
The patch is against:

git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git next-20250123
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm mm-stable

---
 arch/s390/mm/pgalloc.c | 3 +++
 1 file changed, 3 insertions(+)
arch/s390/mm/pgalloc.c +3 -0
index a4e761902093..d33f55b7ee98 100644
@@ -88,12 +88,14 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)
 		if (unlikely(!p4d))
 			goto err_p4d;
 		crst_table_init(p4d, _REGION2_ENTRY_EMPTY);
+		pagetable_p4d_ctor(virt_to_ptdesc(p4d));
 	}
 	if (end > _REGION1_SIZE) {
 		pgd = crst_table_alloc(mm);
 		if (unlikely(!pgd))
 			goto err_pgd;
 		crst_table_init(pgd, _REGION1_ENTRY_EMPTY);
+		pagetable_pgd_ctor(virt_to_ptdesc(pgd));
 	}
 
 	spin_lock_bh(&mm->page_table_lock);
@@ -130,6 +132,7 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)
 	return 0;
 
 err_pgd:
+	pagetable_dtor(virt_to_ptdesc(p4d));
 	crst_table_free(mm, p4d);
 err_p4d:
 	return -ENOMEM;
Signature · 1 line
2.45.2
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 47 lines · On Thu, 23 Jan 2025 17:03:49 +0100 Alexander Gordeev <[email protected]>…

Commit 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table") misses the call to pagetable_p4d_ctor() against a newly allocated P4D table in crst_table_upgrade();

Commit 68c601de75d8 ("mm: introduce ctor/dtor at PGD level") misses the call to pagetable_pgd_ctor() against a newly allocated PGD and the call to pagetable_dtor() against a newly allocated P4D that is about to be freed on crst_table_upgrade() PGD upgrade fail path.

The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.

Fixes 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table")
Fixes 68c601de75d8 ("mm: introduce ctor/dtor at PGD level")

---
The patch is against:

git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git next-20250123
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm mm-stable

---
 arch/s390/mm/pgalloc.c | 3 +++
 1 file changed, 3 insertions(+)
arch/s390/mm/pgalloc.c +2 -0
index a4e761902093..d33f55b7ee98 100644
@@ -88,12 +88,14 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)
 		if (unlikely(!p4d))
 			goto err_p4d;
 		crst_table_init(p4d, _REGION2_ENTRY_EMPTY);
+		pagetable_p4d_ctor(virt_to_ptdesc(p4d));
 	}
 	if (end > _REGION1_SIZE) {
 		pgd = crst_table_alloc(mm);
 		if (unlikely(!pgd))
 			goto err_pgd;
 		crst_table_init(pgd, _REGION1_ENTRY_EMPTY);
+		pagetable_pgd_ctor(virt_to_ptdesc(pgd));
 	}
 
 	spin_lock_bh(&mm->page_table_lock);

arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 138

Quoted 7 lines
arch/s390/mm/pgalloc.c +1 -0
@@ -130,6 +132,7 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)
 	return 0;
 
 err_pgd:
+	pagetable_dtor(virt_to_ptdesc(p4d));
 	crst_table_free(mm, p4d);
 err_p4d:
 	return -ENOMEM;

Thanks, looks good, and nice catch.

Acked-by on v1 1/1
#2 lore raw
In reply to Alexander Gordeev
Quoted 18 lines · Alexander Gordeev

Commit 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table") misses the call to pagetable_p4d_ctor() against a newly allocated P4D table in crst_table_upgrade();

Commit 68c601de75d8 ("mm: introduce ctor/dtor at PGD level") misses the call to pagetable_pgd_ctor() against a newly allocated PGD and the call to pagetable_dtor() against a newly allocated P4D that is about to be freed on crst_table_upgrade() PGD upgrade fail path.

The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.

Fixes 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table")
Fixes 68c601de75d8 ("mm: introduce ctor/dtor at PGD level")
Quoted 9 lines

---
The patch is against:

   git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git	next-20250123
   git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm			mm-stable
---
  arch/s390/mm/pgalloc.c | 3 +++
  1 file changed, 3 insertions(+)

Thanks!

#3 lore raw
In reply to Alexander Gordeev
Quoted 18 lines · Alexander Gordeev

Commit 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table") misses the call to pagetable_p4d_ctor() against a newly allocated P4D table in crst_table_upgrade();

Commit 68c601de75d8 ("mm: introduce ctor/dtor at PGD level") misses the call to pagetable_pgd_ctor() against a newly allocated PGD and the call to pagetable_dtor() against a newly allocated P4D that is about to be freed on crst_table_upgrade() PGD upgrade fail path.

The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.

Fixes 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table")
Fixes 68c601de75d8 ("mm: introduce ctor/dtor at PGD level")
Quoted 5 lines

---
The patch is against:

git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git next-20250123
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm mm-stable

Thank you for putting together this patch! I was completely unaware of this "upgrade" path on s390.

arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 91

Quoted 13 lines
---
 arch/s390/mm/pgalloc.c | 3 +++
 1 file changed, 3 insertions(+)
arch/s390/mm/pgalloc.c +1 -0
index a4e761902093..d33f55b7ee98 100644
@@ -88,12 +88,14 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)
 		if (unlikely(!p4d))
 			goto err_p4d;
 		crst_table_init(p4d, _REGION2_ENTRY_EMPTY);
+		pagetable_p4d_ctor(virt_to_ptdesc(p4d));

This block seems to be equivalent to p4d_alloc_one(), maybe it would be preferable to just call p4d_alloc_one() here to avoid further mismatches in the future (and reduce duplication)?

arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 98

Quoted 7 lines
arch/s390/mm/pgalloc.c +1 -0
 	}
 	if (end > _REGION1_SIZE) {
 		pgd = crst_table_alloc(mm);
 		if (unlikely(!pgd))
 			goto err_pgd;
 		crst_table_init(pgd, _REGION1_ENTRY_EMPTY);
+		pagetable_pgd_ctor(virt_to_ptdesc(pgd));

I was hoping this might be equivalent to pgd_alloc() but it does not include a call to crst_table_init(). Since adding it would be apparently undesirable (having read the other thread), it seems reasonable to add the explicit constructor call.

arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 136

Quoted 9 lines
arch/s390/mm/pgalloc.c +1 -0
 	}
 
 	spin_lock_bh(&mm->page_table_lock);
@@ -130,6 +132,7 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)
 	return 0;
 
 err_pgd:
+	pagetable_dtor(virt_to_ptdesc(p4d));
 	crst_table_free(mm, p4d);

Similarly, this could be a call to p4d_free().

- Kevin

Quoted 2 lines
arch/s390/mm/pgalloc.c +0 -0
 err_p4d:
 	return -ENOMEM;
author maintainer
#4 lore raw

On Fri, Jan 24, 2025 at 08:58:07AM +0100, Kevin Brodsky wrote:

Kevin,
...

Quoted 2 lines

Thank you for putting together this patch! I was completely unaware of this "upgrade" path on s390.

...

Quoted 5 lines
Quoted 5 lines
arch/s390/mm/pgalloc.c +0 -0
index a4e761902093..d33f55b7ee98 100644
@@ -88,12 +88,14 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)
Quoted 8 lines

arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 91

Quoted 4 lines
arch/s390/mm/pgalloc.c +1 -0
 		if (unlikely(!p4d))
 			goto err_p4d;
 		crst_table_init(p4d, _REGION2_ENTRY_EMPTY);
+		pagetable_p4d_ctor(virt_to_ptdesc(p4d));

This block seems to be equivalent to p4d_alloc_one(), maybe it would be preferable to just call p4d_alloc_one() here to avoid further mismatches in the future (and reduce duplication)?

Yes, I wanted to do exactly that (together with p4d_free() you noticed), but then the whole thing with open coded pagetable_pgd_ctor() below gets inconsistent. I would prefer to keep it this way as of now.

Quoted 4 lines
Quoted 4 lines
arch/s390/mm/pgalloc.c +0 -0
 	}
 	if (end > _REGION1_SIZE) {
 		pgd = crst_table_alloc(mm);
 		if (unlikely(!pgd))
Quoted 8 lines

arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 98

Quoted 3 lines
arch/s390/mm/pgalloc.c +1 -0
 			goto err_pgd;
 		crst_table_init(pgd, _REGION1_ENTRY_EMPTY);
+		pagetable_pgd_ctor(virt_to_ptdesc(pgd));

I was hoping this might be equivalent to pgd_alloc() but it does not include a call to crst_table_init(). Since adding it would be apparently undesirable (having read the other thread), it seems reasonable to add the explicit constructor call.

We were thinking about a follow-up cleanup that addresses it all, but this patch is a targeted fix to catch up your and Qi Zheng series in the still open merge window.

Quoted 6 lines
Quoted 6 lines
arch/s390/mm/pgalloc.c +0 -0
 	}
 
 	spin_lock_bh(&mm->page_table_lock);
@@ -130,6 +132,7 @@ int crst_table_upgrade(struct mm_struct *mm, unsigned long end)
 	return 0;
 
Quoted 10 lines

arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 136

Quoted 3 lines
arch/s390/mm/pgalloc.c +1 -0
 err_pgd:
+	pagetable_dtor(virt_to_ptdesc(p4d));
 	crst_table_free(mm, p4d);

Similarly, this could be a call to p4d_free().

- Kevin

Quoted 2 lines
arch/s390/mm/pgalloc.c +0 -0
 err_p4d:
 	return -ENOMEM;

Thanks for the review!

Reviewed-by
#5 lore raw
Quoted 11 lines · Alexander Gordeev
Quoted 11 lines

arch/s390/mm/pgalloc.c int crst_table_upgrade(struct mm_struct *mm, unsigned long end) · line 98

Quoted 7 lines
arch/s390/mm/pgalloc.c +1 -0
 	}
 	if (end > _REGION1_SIZE) {
 		pgd = crst_table_alloc(mm);
 		if (unlikely(!pgd))
 			goto err_pgd;
 		crst_table_init(pgd, _REGION1_ENTRY_EMPTY);
+		pagetable_pgd_ctor(virt_to_ptdesc(pgd));

I was hoping this might be equivalent to pgd_alloc() but it does not include a call to crst_table_init(). Since adding it would be apparently undesirable (having read the other thread), it seems reasonable to add the explicit constructor call.

Quoted 3 lines

We were thinking about a follow-up cleanup that addresses it all, but this patch is a targeted fix to catch up your and Qi Zheng series in the still open merge window.

Sure that's fair, this patch does fix the issue and the cleanup can wait until the next cycle.

- Kevin

maintainer
#6 lore raw
In reply to Kevin Brodsky
Quoted 22 lines · Kevin Brodsky
Quoted 21 lines · Alexander Gordeev

Commit 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table") misses the call to pagetable_p4d_ctor() against a newly allocated P4D table in crst_table_upgrade();

Commit 68c601de75d8 ("mm: introduce ctor/dtor at PGD level") misses the call to pagetable_pgd_ctor() against a newly allocated PGD and the call to pagetable_dtor() against a newly allocated P4D that is about to be freed on crst_table_upgrade() PGD upgrade fail path.

The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.

Fixes 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table")
Fixes 68c601de75d8 ("mm: introduce ctor/dtor at PGD level")

---
The patch is against:

Quoted 5 lines
Quoted 2 lines

git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git next-20250123
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm mm-stable

Thank you for putting together this patch! I was completely unaware of this "upgrade" path on s390.

This whole thing is even worse than it looks after this patch. With page table upgrade taken into account we still have the oddity that e.g. a previous pgd becomes a pud or p4d, which means that ctor and dtor functions might be called for different levels for the same page table. As of now this is ok, since they do all the same.

As a quick fix this patch is ok, and most likely it will be ok for a long time, but it doesn't give me good feeling :)

In addition, looking at [1] where page table accounting was introduced: it is really meant to reflect the memory consumption used by page tables. This might work for nearly all architectures which have the same page table size for every level; but on s390 the lowest level comes with 4kb page tables while all other levels come with 16kb page tables.

Therefore at least on s390 you really can't tell how much memory is consumed by page tables by only looking at nr_page_table_pages. It _might_ make sense to introduce a factor of four for page table accounting for higher levels, so those numbers make at least some sense; but not sure about that.

[1] https://lore.kernel.org/all/[email protected]/T/#mb6c6f2a84ded27cd9b3d54140dde1d5a75c74735

maintainer
#7 lore raw
Quoted 7 lines · Heiko Carstens

On Fri, Jan 24, 2025 at 08:58:07AM +0100, Kevin Brodsky wrote:
In addition, looking at [1] where page table accounting was introduced: it is really meant to reflect the memory consumption used by page tables. This might work for nearly all architectures which have the same page table size for every level; but on s390 the lowest level comes with 4kb page tables while all other levels come with 16kb page tables.

Quoted 5 lines

Therefore at least on s390 you really can't tell how much memory is consumed by page tables by only looking at nr_page_table_pages. It _might_ make sense to introduce a factor of four for page table accounting for higher levels, so those numbers make at least some sense; but not sure about that.

Ah, this is actually not true at all, since we have

static inline void __lruvec_stat_add_folio(struct folio *folio,
enum node_stat_item idx)

{
	__lruvec_stat_mod_folio(folio, idx, folio_nr_pages(folio));
}

which will do exactly what we want. So this part is not a problem.

#8 lore raw
In reply to Heiko Carstens
Quoted 27 lines · Heiko Carstens
Quoted 22 lines · Kevin Brodsky
Quoted 21 lines · Alexander Gordeev

Commit 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table") misses the call to pagetable_p4d_ctor() against a newly allocated P4D table in crst_table_upgrade();

Commit 68c601de75d8 ("mm: introduce ctor/dtor at PGD level") misses the call to pagetable_pgd_ctor() against a newly allocated PGD and the call to pagetable_dtor() against a newly allocated P4D that is about to be freed on crst_table_upgrade() PGD upgrade fail path.

The missed constructors and destructor break (at least) the page table accounting when a process memory space is upgraded.

Fixes 78966b550289 ("s390: pgtable: add statistics for PUD and P4D level page table")
Fixes 68c601de75d8 ("mm: introduce ctor/dtor at PGD level")

---
The patch is against:

Quoted 4 lines
Quoted 2 lines

git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git next-20250123
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm mm-stable

Thank you for putting together this patch! I was completely unaware of this "upgrade" path on s390.

Quoted 8 lines

This whole thing is even worse than it looks after this patch. With page table upgrade taken into account we still have the oddity that e.g. a previous pgd becomes a pud or p4d, which means that ctor and dtor functions might be called for different levels for the same page table. As of now this is ok, since they do all the same.

As a quick fix this patch is ok, and most likely it will be ok for a long time, but it doesn't give me good feeling :)

This could clearly cause trouble if the pud/p4d/pgd constructors start to diverge. In principle this could be addressed when upgrading by calling the dtor on the old pgd and then the ctor on the new pud/p4d. That said since there is now only one dtor, and the ctor are all the same, it's probably good enough.

I'll mention in passing that my own series (that introduced PGD-level ctor/dtor) really aimed at supporting the kpkeys page table protection RFC [1], where hooks are inserted into the ctor/dtor to modify the pkey page tables are mapped with. No idea if that would work with this sort of upgrade path, but since s390 does not support pkeys this is just a theoretical issue :)

- Kevin

[1]
https://lore.kernel.org/linux-hardening/[email protected]/