Skip to contentSkip to conversation

x86/mm/pat: CPA fixes

PATCH Applied Mike Rapoport 12 participants 66 messages Aug 13 – Sep 9

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v2 cover x86/mm/pat: CPA fixes

The first three patches are urgent, the third patch fixes BUG() reported y several people and it depends on the first two.

There were no bug reports that the last two patches fix because bug manifestations won't yell at users.

TL;DR version:

There are a couple of CPA fixes floating around:

Denis Lunev fixed races between split and collapse of the large mappings:

https://lore.kernel.org/all/[email protected]

Lorenzo Stoakes fixed UAF caused by races between CPA and ptdump:

https://lore.kernel.org/all/[email protected]

and an issue with stale page tables in IOMMU:

https://lore.kernel.org/all/[email protected]

Mike Rapoport fixed a check of RW attribute in lookup_address_in_pgd_attr() used for the verification of RWX:

https://lore.kernel.org/all/[email protected]

Pedro Falcato closed a race between text poking and collapse of large pages:

https://lore.kernel.org/all/anCK3eWFMwZqq5ka@pedro-suse

Some of the fixes got merged into x86 tree, some of them got merged into mm tree and some are still hanging in the air.

The changes here are collected from all these fixes into a single coherent set on top of tip/x86/mm:

* fix for races between CPA and ptdump causing UAF
* update to the fix of the race between split and collapse of large mappings
* fix for races between CPA and vmalloc_to_page() in text poking
* fix for stale page tables in IOMMU
* fix for effective RW computation in lookup_address_in_pgd_attr()

---
v2 changes:
* rebased on the current tip/x86/mm that includes peterz's changes for DEBUG_PAGEALLOC
* added fix for CPA vs text poking race

v1: https://patch.msgid.link/[email protected]

---
Lorenzo Stoakes (ARM) (3):
x86/mm/pat: acquire init_mm write lock on collapse to avoid UAF x86/mm/pat: acquire init_mm read lock on attribute change to avoid UAF x86/mm/pat: allocate split page tables as kernel page tables

Mike Rapoport (Microsoft) (1):
x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()

Pedro Falcato (1):
x86/alternative: exclude text poking against change_page_attr()

 arch/x86/kernel/alternative.c | 39 ++++++++++++++++++++++++---
 arch/x86/mm/pat/set_memory.c  | 61 +++++++++++++++++++++++++++++++------------
 include/linux/mmap_lock.h     |  2 ++
 3 files changed, 83 insertions(+), 19 deletions(-)

---
base-commit: 7da514d819a0afb148634aac92b3d190f34947c3
change-id: 20260727-cpa-fixes-d3c73c075672

--
Sincerely yours,
Mike.

#1 lore raw

PATCH v2 1/5 x86/mm/pat: acquire init_mm write lock on collapse to avoid UAF

From: "Lorenzo Stoakes (ARM)" <[email protected]>

x86 implements page attribute modification using its Change Page Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page attributes, and as part of that logic manipulates kernel page tables.

Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation") ranges of kernel page table entries can be collapsed into huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed entries previously pointed to, and it does so without any relevant locks being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and bpf when manipulating executable mappings.

However, this is problematic for ptdump as it walks ranges it does not own and thus runs the risk of a use-after-free on page tables freed underneath it.

In addition, concurrent CPA collapse operations are possible which can also cause races.

Resolve the issue by acquiring the mmap write lock on init_mm across the whole operation.

It is safe to acquire a sleeping lock as all the callers invoke set_memory_rox() from process context and in any case, change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a mutex, disallowing atomic context here.

---
 arch/x86/mm/pat/set_memory.c | 15 ++++++++++++++-
 include/linux/mmap_lock.h    |  2 ++
 2 files changed, 16 insertions(+), 1 deletion(-)
arch/x86/mm/pat/set_memory.c +14 -1
index d8d057f44417..c18b887ee4c2 100644
@@ -22,6 +22,7 @@
 #include <linux/cc_platform.h>
 #include <linux/set_memory.h>
 #include <linux/memregion.h>
+#include <linux/cleanup.h>
 
 #include <asm/e820/api.h>
 #include <asm/processor.h>
@@ -409,7 +410,7 @@ static void __cpa_flush_tlb(void *data)
 
 static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
 
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 {
 	unsigned long start, addr, end;
 	struct ptdesc *ptdesc, *tmp;
@@ -443,6 +444,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
 	}
 }
 
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{
+	/*
+	 * Take the mmap write lock on init_mm to:
+	 * - Avoid a use-after-free if raced by ptdump (which takes its own
+	 *   write lock on init_mm).
+	 * - Serialise concurrent CPA walkers.
+	 */
+	scoped_guard(mmap_write_lock, &init_mm)
+		__cpa_collapse_large_pages(cpa);
+}
+
 static void cpa_flush(struct cpa_data *cpa, int cache)
 {
 	unsigned int i;
include/linux/mmap_lock.h +2 -0
index 04b8f61ece5d..f4ceb968aeb3 100644
@@ -621,6 +621,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
 
 DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
 	     mmap_read_lock(_T), mmap_read_unlock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+	     mmap_write_lock(_T), mmap_write_unlock(_T))
 
 static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
 {
Signature · 1 line
2.53.0
#2 lore raw

[tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     be4f4ab413d15e2b44f6bcda3b607eb707a7712e
Gitweb:        https://git.kernel.org/tip/be4f4ab413d15e2b44f6bcda3b607eb707a7712e
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:24 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:14:58 -07:00

x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

x86 implements page attribute modification using its Change Page Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page attributes, and as part of that logic manipulates kernel page tables.

Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation") ranges of kernel page table entries can be collapsed into huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed entries previously pointed to, and it does so without any relevant locks being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and bpf when manipulating executable mappings.

However, this is problematic for ptdump as it walks ranges it does not own and thus runs the risk of a use-after-free on page tables freed underneath it.

In addition, concurrent CPA collapse operations are possible which can also cause races.

Resolve the issue by acquiring the mmap write lock on init_mm across the whole operation.

It is safe to acquire a sleeping lock as all the callers invoke set_memory_rox() from process context and in any case, change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a mutex, disallowing atomic context here.

---
 include/linux/mmap_lock.h | 2 ++
 1 file changed, 2 insertions(+)
include/linux/mmap_lock.h +2 -0
index bec0eab..b8a13b8 100644
@@ -630,6 +630,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
 DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
 	     mmap_read_lock(_T), mmap_read_unlock(_T))
 DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+	     mmap_write_lock(_T), mmap_write_unlock(_T))
 
 static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
 {
#3 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

Quoted 8 lines · On 01. 09. 26, 0:27, tip-bot2 for Lorenzo Stoakes (ARM) wrote:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     be4f4ab413d15e2b44f6bcda3b607eb707a7712e
Gitweb:        https://git.kernel.org/tip/be4f4ab413d15e2b44f6bcda3b607eb707a7712e
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:24 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:14:58 -07:00

The committed patch to tip is bogus. It contains only the guard definition.

Quoted 74 lines

x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

x86 implements page attribute modification using its Change Page Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page attributes, and as part of that logic manipulates kernel page tables.

Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation") ranges of kernel page table entries can be collapsed into huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed entries previously pointed to, and it does so without any relevant locks being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and bpf when manipulating executable mappings.

However, this is problematic for ptdump as it walks ranges it does not own and thus runs the risk of a use-after-free on page tables freed underneath it.

In addition, concurrent CPA collapse operations are possible which can also cause races.

Resolve the issue by acquiring the mmap write lock on init_mm across the whole operation.

It is safe to acquire a sleeping lock as all the callers invoke set_memory_rox() from process context and in any case, change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a mutex, disallowing atomic context here.

---
  include/linux/mmap_lock.h | 2 ++
  1 file changed, 2 insertions(+)
include/linux/mmap_lock.h +2 -0
index bec0eab..b8a13b8 100644
@@ -630,6 +630,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
  DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
  	     mmap_read_lock(_T), mmap_read_unlock(_T))
  DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+	     mmap_write_lock(_T), mmap_write_unlock(_T))
 
  static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
  {
Signature · 2 lines
js
suse labs
#4 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

Quoted 11 lines · Jiri Slaby
Quoted 8 lines · On 01. 09. 26, 0:27, tip-bot2 for Lorenzo Stoakes (ARM) wrote:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     be4f4ab413d15e2b44f6bcda3b607eb707a7712e
Gitweb:        https://git.kernel.org/tip/be4f4ab413d15e2b44f6bcda3b607eb707a7712e
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:24 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:14:58 -07:00

The committed patch to tip is bogus. It contains only the guard definition.

Dave - you've somehow applied this patch completely incorrectly to x86/urgent, I'm not happy with this going to Linus in this form :/

Now the commit message and the actual patch are completely mismatched.

I'm not sure how tip resolves issues like these but is it possible to replace this with the actual patch that was submitted please?

Thanks.

Quoted 43 lines
Quoted 43 lines

x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

x86 implements page attribute modification using its Change Page Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page attributes, and as part of that logic manipulates kernel page tables.

Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation") ranges of kernel page table entries can be collapsed into huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed entries previously pointed to, and it does so without any relevant locks being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and bpf when manipulating executable mappings.

However, this is problematic for ptdump as it walks ranges it does not own and thus runs the risk of a use-after-free on page tables freed underneath it.

In addition, concurrent CPA collapse operations are possible which can also cause races.

Resolve the issue by acquiring the mmap write lock on init_mm across the whole operation.

It is safe to acquire a sleeping lock as all the callers invoke set_memory_rox() from process context and in any case, change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a mutex, disallowing atomic context here.

Quoted 12 lines
Quoted 4 lines
Fixes 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
Quoted 8 lines

It renders all of these tags completly incorrect too.

Quoted 6 lines
Quoted 6 lines
---
  include/linux/mmap_lock.h | 2 ++
  1 file changed, 2 insertions(+)
Quoted 10 lines

include/linux/mmap_lock.h static inline void mmap_read_unlock(struct mm_struct *mm) · line 634

Quoted 10 lines
include/linux/mmap_lock.h +2 -0
index bec0eab..b8a13b8 100644
@@ -630,6 +630,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
  DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
  	     mmap_read_lock(_T), mmap_read_unlock(_T))
  DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+	     mmap_write_lock(_T), mmap_write_unlock(_T))

Yeah I meant obviously this isn't what the patch is.

Quoted 8 lines

include/linux/mmap_lock.h static inline void mmap_read_unlock(struct mm_struct *mm)

Quoted 3 lines
include/linux/mmap_lock.h +0 -0
  static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
  {
 

--
js
suse labs

--
Cheers, Lorenzo

#5 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

Quoted 2 lines · Lorenzo Stoakes (ARM)

Dave - you've somehow applied this patch completely incorrectly to x86/urgent, I'm not happy with this going to Linus in this form :/

Me neither, btw.

So, here it is applied:

Quoted 1 line

https://git.kernel.org/pub/scm/linux/kernel/git/daveh/devel.git/log/?h=cpa1

The diffstat exactly matches the mbox that I sucked in. I'm going to launch some tests overnight.

The only things I really munged were a little merge issue with patch 1, a SoB ordering issue in patch 3, and a few scattered rewordings in the commit messages.

Not pushed to x86/urgent, yet.

#6 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

Quoted 5 lines · Dave Hansen
Quoted 2 lines · Lorenzo Stoakes (ARM)

Dave - you've somehow applied this patch completely incorrectly to x86/urgent, I'm not happy with this going to Linus in this form :/

Me neither, btw.

Yeah, sorry to complain so loudly, but just obviously want to make sure these are sorted correctly!

Quoted 7 lines

So, here it is applied:

Quoted 1 line

https://git.kernel.org/pub/scm/linux/kernel/git/daveh/devel.git/log/?h=cpa1

The diffstat exactly matches the mbox that I sucked in. I'm going to launch some tests overnight.

Thanks!

I had a look through and all LGTM :)

Quoted 4 lines

The only things I really munged were a little merge issue with patch 1, a SoB ordering issue in patch 3, and a few scattered rewordings in the commit messages.

That's all good of course!

Quoted 2 lines

Not pushed to x86/urgent, yet.

Ack, understood that you want to beat them about a bit with some tests, let me know if there's anything else you need on these!

--
Cheers, Lorenzo

#7 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

In reply to Dave Hansen
Quoted 10 lines · Dave Hansen
Quoted 2 lines · Lorenzo Stoakes (ARM)

Dave - you've somehow applied this patch completely incorrectly to x86/urgent, I'm not happy with this going to Linus in this form :/

Me neither, btw.

So, here it is applied:

Quoted 1 line

https://git.kernel.org/pub/scm/linux/kernel/git/daveh/devel.git/log/?h=cpa1

Quoted 8 lines

The diffstat exactly matches the mbox that I sucked in. I'm going to launch some tests overnight.

The only things I really munged were a little merge issue with patch 1, a SoB ordering issue in patch 3, and a few scattered rewordings in the commit messages.

Not pushed to x86/urgent, yet.

I have to say I'm confused with today's batch of tip-bot emails saying Ingo applied these to x86/mm :/

Signature · 2 lines
Sincerely yours,
Mike.
#8 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

Quoted 14 lines · Mike Rapoport
Quoted 13 lines · Dave Hansen
Quoted 2 lines · Lorenzo Stoakes (ARM)

Dave - you've somehow applied this patch completely incorrectly to x86/urgent, I'm not happy with this going to Linus in this form :/

Me neither, btw.

So, here it is applied:

Quoted 1 line

https://git.kernel.org/pub/scm/linux/kernel/git/daveh/devel.git/log/?h=cpa1

The diffstat exactly matches the mbox that I sucked in. I'm going to launch some tests overnight.

Quoted 8 lines
Quoted 5 lines

The only things I really munged were a little merge issue with patch 1, a SoB ordering issue in patch 3, and a few scattered rewordings in the commit messages.

Not pushed to x86/urgent, yet.

I have to say I'm confused with today's batch of tip-bot emails saying Ingo applied these to x86/mm :/

Since I don't seem to get replies to mails on this topic, I can only _guess_ :), but:

* I see patches in x86/urgent that aren't in x86/mm, so that rules out x86/mm being a stepping stone in the process.

* x86/urgent has had more patches added to it, for series sent after this one, which fix bugs from 2024, 2025 etc. So that's all working as expected.

* We've already highlighted that this series fixes bugs users are seeing in production, so I don't see why it'd be deprioritised.

* It's coming form a private dev repo/branch so I can't imagine it's a wayward script (but maybe?)

* The patches have been in linux-next for a while (maybe Dave/Ingo wanted them there longer? They were there for weeks before the ->x86 stuff though).

Perhaps the reported bug that you've now fixed was the hold up? I guess who knows :)

So yeah, I'm also rather confused :)

Ah well.

I guess taking them through the mm tree wouldn't really be sensible/the right thing to do at this stage, so I guess let's hope they land for 7.4.

Quoted 4 lines

--
Sincerely yours,
Mike.

--
Cheers, Lorenzo

#9 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

In reply to Mike Rapoport
Quoted 2 lines · Mike Rapoport

I have to say I'm confused with today's batch of tip-bot emails saying Ingo applied these to x86/mm :/

Just a reminder that we have x86/urgent which will be going to Linus like today. There's also x86/mm which goes up in the next merge window..

The "Fix effective RW computation" patch is getting broken out and is going to wait for the next merge window. We don't want it sitting in mainline generating warnings, especially false positives like the one this weekend. That's not to disparage the patch; it's overall the right thing to do.

Sound good?

#10 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

Quoted 6 lines · Dave Hansen
Quoted 2 lines · Mike Rapoport

I have to say I'm confused with today's batch of tip-bot emails saying Ingo applied these to x86/mm :/

Just a reminder that we have x86/urgent which will be going to Linus like today. There's also x86/mm which goes up in the next merge window..

Quoted 7 lines

The "Fix effective RW computation" patch is getting broken out and is going to wait for the next merge window. We don't want it sitting in mainline generating warnings, especially false positives like the one this weekend. That's not to disparage the patch; it's overall the right thing to do.

Sound good?

... and no, that isn't the current state of x86/mm and x86/urgent. But that's where I _want_ to end up.

#11 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

In reply to Dave Hansen
Quoted 6 lines · Dave Hansen
Quoted 2 lines · Mike Rapoport

I have to say I'm confused with today's batch of tip-bot emails saying Ingo applied these to x86/mm :/

Just a reminder that we have x86/urgent which will be going to Linus like today. There's also x86/mm which goes up in the next merge window..

However x86/urgent (at least as visible [1]) doesn't contain anything from this series, and x86/mm has everything except "Fix effective RW computation".

But tip/urgent does merge x86/mm so hopefully it means it's really going to Linus now-ish (and not next merge window)? It just doesn't have "urgent" in the branch name?

Quoted 5 lines

The "Fix effective RW computation" patch is getting broken out and is going to wait for the next merge window. We don't want it sitting in mainline generating warnings, especially false positives like the one this weekend. That's not to disparage the patch; it's overall the right thing to do.

So that means adding it to x86/mm after the current one is sent as urgent?

Just wanted to double check, thanks.
Vlastimil

[1] https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/

Quoted 1 line

Sound good?

#12 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

Quoted 8 lines · Vlastimil Babka (SUSE)
Quoted 6 lines · Dave Hansen
Quoted 2 lines · Mike Rapoport

I have to say I'm confused with today's batch of tip-bot emails saying Ingo applied these to x86/mm :/

Just a reminder that we have x86/urgent which will be going to Linus like today. There's also x86/mm which goes up in the next merge window..

Quoted 6 lines

However x86/urgent (at least as visible [1]) doesn't contain anything from this series, and x86/mm has everything except "Fix effective RW computation".

But tip/urgent does merge x86/mm so hopefully it means it's really going to Linus now-ish (and not next merge window)? It just doesn't have "urgent" in the branch name?

No, tip/urgent doesn't generally merge x86/mm.

Quoted 7 lines
Quoted 5 lines

The "Fix effective RW computation" patch is getting broken out and is going to wait for the next merge window. We don't want it sitting in mainline generating warnings, especially false positives like the one this weekend. That's not to disparage the patch; it's overall the right thing to do.

So that means adding it to x86/mm after the current one is sent as urgent?

There are a bunch of things in play:

1. The non-mm x86/urgent things
2. The CPA changes that cause actual problem
3. The CPA R^X permission check fix
4. The _PAGE_DIRTY dropping fix (separate, not CPA related)

The mm stuff is currently blocking the other stuff.

I just sent all the non-mm stuff up in a pull request to Linus.

My plan is to queue #2 and #4 back into x86/urgent once that gets pulled. That will get the functional fixes up to Linus as soon.

I'll *then* queue #3 (the permission check) into x86/mm where it can stew for the next ~6 weeks. It will go upstream for the next merge window.

#13 lore raw

Update on CPA fixes, x86/urgent and x86/mm

OK, so Linus pulled the x86/urgent fixes I sent earlier today. I've staged everything else. Here's an explanation for where that leaves the other x86 fixes.

==== x86/urgent ====

x86/urgent has 5 urgent fixes in it. There are 4 that are CPA-related. These fix some use-after-free issues that have been blocking mm folks from making forward progress on some other features they're working on. So they're impacting real people. They're not new bugs, but they're annoying enough not to want to wait until the merge window:

x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF x86/mm/pat: Acquire init_mm read lock on attribute changes... x86/mm/pat: Allocate split page tables as kernel page tables x86/alternatives: Exclude text poking against change_page_attr()

The other one is old as well, but it's actually causing user data loss. It's a relatively obscure set of features and requires heavy reclaim to trigger, which explains why it wasn't found for so long. But the side effects are rather nasty, so it needs fixing upstream now.

x86/mm: Fix user-space data loss with MADV_FREE and THP

====== x86/mm ======

x86/mm now has a single commit:

x86/mm/pat: Fix effective RW computation in ...

But, being a bug fix for a bug detector, folks have bisected a "regression" to it:

	https://lore.kernel.org/all/20260905044253.GA3816371@ax162/

It is a great and very welcome bug report, but appears to be a false-positive. Rather than deal with a pile of these in Linus's tree, keeping this in x86/mm and getting it linux-next exposure for the coming weeks seems prudent.

I hope this makes sense to everyone.

#14 lore raw

Re: Update on CPA fixes, x86/urgent and x86/mm

Quoted 7 lines · Dave Hansen

OK, so Linus pulled the x86/urgent fixes I sent earlier today. I've staged everything else. Here's an explanation for where that leaves the other x86 fixes.

==== x86/urgent ====

x86/urgent has 5 urgent fixes in it. There are 4 that are CPA-related.

Note that the rebase you did of x86/mm onto x86/urgent added the 5 MM fixes with an incorrect SOB chain - I've rebased x86/urgent to fix this up. No other changes, so if there are no other issues I this we can send this to Linus later today, and thus give it an additional 2-3 days of testing in upstream until -rc3.

Thanks,

Ingo

#15 lore raw

Re: Update on CPA fixes, x86/urgent and x86/mm

Quoted 7 lines · Ingo Molnar
Quoted 4 lines · Dave Hansen

OK, so Linus pulled the x86/urgent fixes I sent earlier today. I've staged everything else. Here's an explanation for where that leaves the other x86 fixes.

Quoted 8 lines
Quoted 3 lines

==== x86/urgent ====

x86/urgent has 5 urgent fixes in it. There are 4 that are CPA-related.

Note that the rebase you did of x86/mm onto x86/urgent added the 5 MM fixes with an incorrect SOB chain - I've rebased x86/urgent to fix this up. No other changes, so if there are no other issues I this we can send this to Linus later today, and thus give it

^think

Quoted 1 line

an additional 2-3 days of testing in upstream until -rc3.

When morning tea has not kicked in yet. ;-)

Thanks,

Ingo

#16 lore raw

[tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     4cfad2657c7c87b1172a9c343b74cf59b3769873
Gitweb:        https://git.kernel.org/tip/4cfad2657c7c87b1172a9c343b74cf59b3769873
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:24 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Tue, 01 Sep 2026 14:31:49 -07:00

x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

x86 implements page attribute modification using its Change Page Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page attributes, and as part of that logic manipulates kernel page tables.

Since commit 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation") ranges of kernel page table entries can be collapsed into huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed entries previously pointed to, and it does so without any relevant locks being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and bpf when manipulating executable mappings.

However, this is problematic for ptdump as it walks ranges it does not own and thus runs the risk of a use-after-free on page tables freed underneath it.

In addition, concurrent CPA collapse operations are possible which can also cause races.

Resolve the issue by acquiring the mmap write lock on init_mm across the whole operation.

It is safe to acquire a sleeping lock as all the callers invoke set_memory_rox() from process context and in any case, change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a mutex, disallowing atomic context here.

---
 arch/x86/mm/pat/set_memory.c | 15 ++++++++++++++-
 include/linux/mmap_lock.h    |  2 ++
 2 files changed, 16 insertions(+), 1 deletion(-)
arch/x86/mm/pat/set_memory.c +14 -1
index c38faf3..4abddd7 100644
@@ -22,6 +22,7 @@
 #include <linux/cc_platform.h>
 #include <linux/set_memory.h>
 #include <linux/memregion.h>
+#include <linux/cleanup.h>
 
 #include <asm/e820/api.h>
 #include <asm/processor.h>
@@ -409,7 +410,7 @@ static void __cpa_flush_tlb(void *data)
 
 static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
 
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 {
 	unsigned long start, addr, end;
 	struct ptdesc *ptdesc, *tmp;
@@ -443,6 +444,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
 	}
 }
 
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{
+	/*
+	 * Take the mmap write lock on init_mm to:
+	 * - Avoid a use-after-free if raced by ptdump (which takes its own
+	 *   write lock on init_mm).
+	 * - Serialise concurrent CPA walkers.
+	 */
+	scoped_guard(mmap_write_lock, &init_mm)
+		__cpa_collapse_large_pages(cpa);
+}
+
 static void cpa_flush(struct cpa_data *cpa, int cache)
 {
 	unsigned int i;
include/linux/mmap_lock.h +2 -0
index bec0eab..b8a13b8 100644
@@ -630,6 +630,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
 DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
 	     mmap_read_lock(_T), mmap_read_unlock(_T))
 DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+	     mmap_write_lock(_T), mmap_write_unlock(_T))
 
 static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
 {
#17 lore raw

[tip: x86/mm] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

In reply to Mike Rapoport
The following commit has been merged into the x86/mm branch of tip:

The following commit has been merged into the x86/mm branch of tip:

Commit-ID:     47b439d627718162b04efe241418775153fc9193
Gitweb:        https://git.kernel.org/tip/47b439d627718162b04efe241418775153fc9193
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:24 +03:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Tue, 08 Sep 2026 09:06:33 +02:00

x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

x86 implements page attribute modification using its Change Page Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page attributes, and as part of that logic manipulates kernel page tables.

Since commit:

41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")

ranges of kernel page table entries can be collapsed into huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed entries previously pointed to, and it does so without any relevant locks being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and BPF when manipulating executable mappings.

However, this is problematic for ptdump as it walks ranges it does not own and thus runs the risk of a use-after-free on page tables freed underneath it.

In addition, concurrent CPA collapse operations are possible which can also cause races.

Resolve the issue by acquiring the mmap write lock on init_mm across the whole operation.

It is safe to acquire a sleeping lock as all the callers invoke set_memory_rox() from process context and in any case, change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a mutex, disallowing atomic context here.

---
 arch/x86/mm/pat/set_memory.c | 15 ++++++++++++++-
 include/linux/mmap_lock.h    |  2 ++
 2 files changed, 16 insertions(+), 1 deletion(-)
arch/x86/mm/pat/set_memory.c +14 -1
index c38faf3..4abddd7 100644
@@ -22,6 +22,7 @@
 #include <linux/cc_platform.h>
 #include <linux/set_memory.h>
 #include <linux/memregion.h>
+#include <linux/cleanup.h>
 
 #include <asm/e820/api.h>
 #include <asm/processor.h>
@@ -409,7 +410,7 @@ static void __cpa_flush_tlb(void *data)
 
 static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
 
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 {
 	unsigned long start, addr, end;
 	struct ptdesc *ptdesc, *tmp;
@@ -443,6 +444,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
 	}
 }
 
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{
+	/*
+	 * Take the mmap write lock on init_mm to:
+	 * - Avoid a use-after-free if raced by ptdump (which takes its own
+	 *   write lock on init_mm).
+	 * - Serialise concurrent CPA walkers.
+	 */
+	scoped_guard(mmap_write_lock, &init_mm)
+		__cpa_collapse_large_pages(cpa);
+}
+
 static void cpa_flush(struct cpa_data *cpa, int cache)
 {
 	unsigned int i;
include/linux/mmap_lock.h +2 -0
index bec0eab..b8a13b8 100644
@@ -630,6 +630,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
 DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
 	     mmap_read_lock(_T), mmap_read_unlock(_T))
 DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+	     mmap_write_lock(_T), mmap_write_unlock(_T))
 
 static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
 {
#18 lore raw

[tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     e65d3f0d78b79cf933b8755417846b172bc8a05d
Gitweb:        https://git.kernel.org/tip/e65d3f0d78b79cf933b8755417846b172bc8a05d
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:24 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Tue, 08 Sep 2026 15:43:06 -07:00

x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

x86 implements page attribute modification using its Change Page Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page attributes, and as part of that logic manipulates kernel page tables.

Since commit:

41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")

ranges of kernel page table entries can be collapsed into huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed entries previously pointed to, and it does so without any relevant locks being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and BPF when manipulating executable mappings.

However, this is problematic for ptdump as it walks ranges it does not own and thus runs the risk of a use-after-free on page tables freed underneath it.

In addition, concurrent CPA collapse operations are possible which can also cause races.

Resolve the issue by acquiring the mmap write lock on init_mm across the whole operation.

It is safe to acquire a sleeping lock as all the callers invoke set_memory_rox() from process context and in any case, change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a mutex, disallowing atomic context here.

---
 arch/x86/mm/pat/set_memory.c | 15 ++++++++++++++-
 include/linux/mmap_lock.h    |  2 ++
 2 files changed, 16 insertions(+), 1 deletion(-)
arch/x86/mm/pat/set_memory.c +14 -1
index c38faf3..4abddd7 100644
@@ -22,6 +22,7 @@
 #include <linux/cc_platform.h>
 #include <linux/set_memory.h>
 #include <linux/memregion.h>
+#include <linux/cleanup.h>
 
 #include <asm/e820/api.h>
 #include <asm/processor.h>
@@ -409,7 +410,7 @@ static void __cpa_flush_tlb(void *data)
 
 static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
 
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 {
 	unsigned long start, addr, end;
 	struct ptdesc *ptdesc, *tmp;
@@ -443,6 +444,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
 	}
 }
 
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{
+	/*
+	 * Take the mmap write lock on init_mm to:
+	 * - Avoid a use-after-free if raced by ptdump (which takes its own
+	 *   write lock on init_mm).
+	 * - Serialise concurrent CPA walkers.
+	 */
+	scoped_guard(mmap_write_lock, &init_mm)
+		__cpa_collapse_large_pages(cpa);
+}
+
 static void cpa_flush(struct cpa_data *cpa, int cache)
 {
 	unsigned int i;
include/linux/mmap_lock.h +2 -0
index bec0eab..b8a13b8 100644
@@ -630,6 +630,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
 DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
 	     mmap_read_lock(_T), mmap_read_unlock(_T))
 DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+	     mmap_write_lock(_T), mmap_write_unlock(_T))
 
 static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
 {
#19 lore raw

[tip: x86/urgent] x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     a1c7570cedd03372812a5b693732880867babbca
Gitweb:        https://git.kernel.org/tip/a1c7570cedd03372812a5b693732880867babbca
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:24 +03:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 09 Sep 2026 08:38:56 +02:00

x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF

x86 implements page attribute modification using its Change Page Attributes (CPA) mechanism.

This tracks properties of ranges such as cache mode through x86 page attributes, and as part of that logic manipulates kernel page tables.

Since commit:

41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")

ranges of kernel page table entries can be collapsed into huge page table entries as part of this logic.

As part of this collapse, it frees the page tables which the collapsed entries previously pointed to, and it does so without any relevant locks being held to preclude concurrent kernel page table walkers.

The only way this code can be reached is if CPA_COLLAPSE is specified, and this is only set in set_memory_rox() via:

set_memory_rox()
-> change_page_attr_set_clr()
-> cpa_flush()
-> cpa_collapse_large_pages()

Notable users of this are execmem and BPF when manipulating executable mappings.

However, this is problematic for ptdump as it walks ranges it does not own and thus runs the risk of a use-after-free on page tables freed underneath it.

In addition, concurrent CPA collapse operations are possible which can also cause races.

Resolve the issue by acquiring the mmap write lock on init_mm across the whole operation.

It is safe to acquire a sleeping lock as all the callers invoke set_memory_rox() from process context and in any case, change_page_attr_set_clr() calls vm_unmap_alias() which ultimately takes a mutex, disallowing atomic context here.

---
 arch/x86/mm/pat/set_memory.c | 15 ++++++++++++++-
 include/linux/mmap_lock.h    |  2 ++
 2 files changed, 16 insertions(+), 1 deletion(-)
arch/x86/mm/pat/set_memory.c +14 -1
index c38faf3..4abddd7 100644
@@ -22,6 +22,7 @@
 #include <linux/cc_platform.h>
 #include <linux/set_memory.h>
 #include <linux/memregion.h>
+#include <linux/cleanup.h>
 
 #include <asm/e820/api.h>
 #include <asm/processor.h>
@@ -409,7 +410,7 @@ static void __cpa_flush_tlb(void *data)
 
 static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
 
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 {
 	unsigned long start, addr, end;
 	struct ptdesc *ptdesc, *tmp;
@@ -443,6 +444,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
 	}
 }
 
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{
+	/*
+	 * Take the mmap write lock on init_mm to:
+	 * - Avoid a use-after-free if raced by ptdump (which takes its own
+	 *   write lock on init_mm).
+	 * - Serialise concurrent CPA walkers.
+	 */
+	scoped_guard(mmap_write_lock, &init_mm)
+		__cpa_collapse_large_pages(cpa);
+}
+
 static void cpa_flush(struct cpa_data *cpa, int cache)
 {
 	unsigned int i;
include/linux/mmap_lock.h +2 -0
index bec0eab..b8a13b8 100644
@@ -630,6 +630,8 @@ static inline void mmap_read_unlock(struct mm_struct *mm)
 DEFINE_GUARD(mmap_read_lock, struct mm_struct *,
 	     mmap_read_lock(_T), mmap_read_unlock(_T))
 DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T))
+DEFINE_GUARD(mmap_write_lock, struct mm_struct *,
+	     mmap_write_lock(_T), mmap_write_unlock(_T))
 
 static inline void mmap_read_unlock_non_owner(struct mm_struct *mm)
 {
#20 lore raw

PATCH v2 2/5 x86/mm/pat: acquire init_mm read lock on attribute change to avoid UAF

In reply to Mike Rapoport

From: "Lorenzo Stoakes (ARM)" <[email protected]>

A previous commit protected us against races between ptdump and CPA collapse, however one still exists between attribute changes and collapse as reported by Denis V. Lunev (linked).

When an attribute change arises, a lockless page table walker obtains a PTE entry, which is later written to via set_pte_atomic():

...
-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()

	-> _lookup_address_cpa()
	-> lookup_address_in_pgd_attr()
	-> [ lockless page table walker ]

-> set_pte_atomic()

There is nothing preventing a concurrent CPA collapse which can free the PTE that was retrieved here, resulting in a use-after-free.

With the mmap write lock taken on init_mm over CPA collapse, we can now resolve this race by acquiring an mmap read lock on init_mm over __change_page_attr_set_clr().

This locks across the whole operation over which the walk and the PTE entry write occurs, solving the race.

It is safe to do this here, as no spinlocks are held upon entry to __change_page_attr_set_clr().

However, the lock must not be held over an allocation, as allocation can trigger reclaim and shrinkers may call into CPA recursively, making deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).

A page table is allocated when a huge page needs to be split:

-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
-> split_large_page()
[ pagetable_alloc() ]
-> __split_large_page()

Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in split_large_page() and track whether this is needed by adding a new 'init_mm_read_locked' flag to struct cpa_data.

This is safe as __split_large_page() (called with locks re-established) revalidates that the page table entry is the same as it was prior to the locks being dropped and __change_page_attr() repeats the entire page table walk whenever a split occurs, so concurrent split and collapse are accounted for.

Concurrent ptdump is also safe as the lock is only dropped over page table allocation during which time the page table has not yet been modified.

The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively operates upon vmalloc ranges, and on x86 only within the module mapping space.

This is important, because some callers directly invoke __change_page_attr_set_clr(), bypassing this lock. However, none of these operate within the module mapping space.

* cpa_process_alias() - a recursive helper called by __change_page_attr_set_clr().
* __set_memory_enc_pgtable() - operates on the direct mapping and (via __vmbus_establish_gpadl()) the vmalloc mapping space.
* __set_pages_[n]p() - called by set_direct_map_[invalid, default, valid]_noflush(), __kernel_map_pages() - operates on the direct map.
* kernel_[un]map_pages_in_pgd() - operates on EFI ranges.

This work is based upon Denis V. Lunev's excellent analysis of the bug with gratitude.

Fixes 41d88484c71c ("x86/mm/pat: restore large ROX pages after fragmentation")
---
 arch/x86/mm/pat/set_memory.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
arch/x86/mm/pat/set_memory.c +11 -2
index c18b887ee4c2..2d04a4bf34aa 100644
@@ -50,7 +50,8 @@ struct cpa_data {
 	unsigned int	flags;
 	unsigned int	force_split		: 1,
 			force_static_prot	: 1,
-			force_flush_all		: 1;
+			force_flush_all		: 1,
+			init_mm_read_locked	: 1;
 	struct page	**pages;
 };
 
@@ -1240,7 +1241,11 @@ static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 	struct ptdesc *ptdesc;
 
 	spin_unlock(&cpa_lock);
+	if (cpa->init_mm_read_locked)
+		mmap_read_unlock(&init_mm);
 	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	if (cpa->init_mm_read_locked)
+		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
 	if (!ptdesc)
 		return -ENOMEM;
@@ -2134,7 +2139,11 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,
 	cpa.curpage = 0;
 	cpa.force_split = force_split;
 
-	ret = __change_page_attr_set_clr(&cpa, 1);
+	/* Avoid race with concurrent CPA collapse. */
+	cpa.init_mm_read_locked = true;
+	scoped_guard(mmap_read_lock, &init_mm)
+		ret = __change_page_attr_set_clr(&cpa, 1);
+	cpa.init_mm_read_locked = false;
 
 	/*
 	 * Check whether we really changed something:
Signature · 1 line
2.53.0
#21 lore raw

[tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     477cf5cd1f698053df7426b7b8d9339e85e00946
Gitweb:        https://git.kernel.org/tip/477cf5cd1f698053df7426b7b8d9339e85e00946
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:25 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:15:42 -07:00

x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

A previous commit protected against races between ptdump and CPA collapse, however one still exists between attribute changes and collapse as reported by Denis V. Lunev (linked).

When an attribute change arises, a lockless page table walker obtains a PTE entry, which is later written to via set_pte_atomic():

...
-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()

	-> _lookup_address_cpa()
	-> lookup_address_in_pgd_attr()
	-> [ lockless page table walker ]

-> set_pte_atomic()

There is nothing preventing a concurrent CPA collapse which can free the PTE that was retrieved here, resulting in a use-after-free.

With the mmap write lock taken on init_mm over CPA collapse, resolve this race by acquiring an mmap read lock on init_mm over __change_page_attr_set_clr().

This locks across the whole operation over which the walk and the PTE entry write occurs, solving the race.

It is safe to do this here, as no spinlocks are held upon entry to __change_page_attr_set_clr().

However, the lock must not be held over an allocation, as allocation can trigger reclaim and shrinkers may call into CPA recursively, making deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).

A page table is allocated when a huge page needs to be split:

-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
-> split_large_page()
[ pagetable_alloc() ]
-> __split_large_page()

Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in split_large_page() and track whether this is needed by adding a new 'init_mm_read_locked' flag to struct cpa_data.

This is safe as __split_large_page() (called with locks re-established) revalidates that the page table entry is the same as it was prior to the locks being dropped and __change_page_attr() repeats the entire page table walk whenever a split occurs, so concurrent split and collapse are accounted for.

Concurrent ptdump is also safe as the lock is only dropped over page table allocation during which time the page table has not yet been modified.

The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively operates upon vmalloc ranges, and on x86 only within the module mapping space.

This is important, because some callers directly invoke __change_page_attr_set_clr(), bypassing this lock. However, none of these operate within the module mapping space.

* cpa_process_alias() - a recursive helper called by __change_page_attr_set_clr().
* __set_memory_enc_pgtable() - operates on the direct mapping and (via __vmbus_establish_gpadl()) the vmalloc mapping space.
* __set_pages_[n]p() - called by set_direct_map_[invalid, default, valid]_noflush(), __kernel_map_pages() - operates on the direct map.
* kernel_[un]map_pages_in_pgd() - operates on EFI ranges.

This work is based upon Denis V. Lunev's excellent analysis of the bug with gratitude.

---
 arch/x86/mm/pat/set_memory.c | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)
arch/x86/mm/pat/set_memory.c +25 -3
index c38faf3..cb5d6d6 100644
@@ -22,6 +22,7 @@
 #include <linux/cc_platform.h>
 #include <linux/set_memory.h>
 #include <linux/memregion.h>
+#include <linux/cleanup.h>
 
 #include <asm/e820/api.h>
 #include <asm/processor.h>
@@ -49,7 +50,8 @@ struct cpa_data {
 	unsigned int	flags;
 	unsigned int	force_split		: 1,
 			force_static_prot	: 1,
-			force_flush_all		: 1;
+			force_flush_all		: 1,
+			init_mm_read_locked	: 1;
 	struct page	**pages;
 };
 
@@ -409,7 +411,7 @@ static void __cpa_flush_tlb(void *data)
 
 static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
 
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 {
 	unsigned long start, addr, end;
 	struct ptdesc *ptdesc, *tmp;
@@ -443,6 +445,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
 	}
 }
 
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{
+	/*
+	 * Take the mmap write lock on init_mm to:
+	 * - Avoid a use-after-free if raced by ptdump (which takes its own
+	 *   write lock on init_mm).
+	 * - Serialise concurrent CPA walkers.
+	 */
+	scoped_guard(mmap_write_lock, &init_mm)
+		__cpa_collapse_large_pages(cpa);
+}
+
 static void cpa_flush(struct cpa_data *cpa, int cache)
 {
 	unsigned int i;
@@ -1227,7 +1241,11 @@ static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 	struct ptdesc *ptdesc;
 
 	spin_unlock(&cpa_lock);
+	if (cpa->init_mm_read_locked)
+		mmap_read_unlock(&init_mm);
 	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	if (cpa->init_mm_read_locked)
+		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
 	if (!ptdesc)
 		return -ENOMEM;
@@ -2121,7 +2139,11 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,
 	cpa.curpage = 0;
 	cpa.force_split = force_split;
 
-	ret = __change_page_attr_set_clr(&cpa, 1);
+	/* Avoid race with concurrent CPA collapse. */
+	cpa.init_mm_read_locked = true;
+	scoped_guard(mmap_read_lock, &init_mm)
+		ret = __change_page_attr_set_clr(&cpa, 1);
+	cpa.init_mm_read_locked = false;
 
 	/*
 	 * Check whether we really changed something:
#22 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

Quoted 135 lines · On 01. 09. 26, 0:27, tip-bot2 for Lorenzo Stoakes (ARM) wrote:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     477cf5cd1f698053df7426b7b8d9339e85e00946
Gitweb:        https://git.kernel.org/tip/477cf5cd1f698053df7426b7b8d9339e85e00946
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:25 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:15:42 -07:00

x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

A previous commit protected against races between ptdump and CPA collapse, however one still exists between attribute changes and collapse as reported by Denis V. Lunev (linked).

When an attribute change arises, a lockless page table walker obtains a PTE entry, which is later written to via set_pte_atomic():

...
-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()

	-> _lookup_address_cpa()
	-> lookup_address_in_pgd_attr()
	-> [ lockless page table walker ]

-> set_pte_atomic()

There is nothing preventing a concurrent CPA collapse which can free the PTE that was retrieved here, resulting in a use-after-free.

With the mmap write lock taken on init_mm over CPA collapse, resolve this race by acquiring an mmap read lock on init_mm over __change_page_attr_set_clr().

This locks across the whole operation over which the walk and the PTE entry write occurs, solving the race.

It is safe to do this here, as no spinlocks are held upon entry to __change_page_attr_set_clr().

However, the lock must not be held over an allocation, as allocation can trigger reclaim and shrinkers may call into CPA recursively, making deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).

A page table is allocated when a huge page needs to be split:

-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
-> split_large_page()
[ pagetable_alloc() ]
-> __split_large_page()

Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in split_large_page() and track whether this is needed by adding a new 'init_mm_read_locked' flag to struct cpa_data.

This is safe as __split_large_page() (called with locks re-established) revalidates that the page table entry is the same as it was prior to the locks being dropped and __change_page_attr() repeats the entire page table walk whenever a split occurs, so concurrent split and collapse are accounted for.

Concurrent ptdump is also safe as the lock is only dropped over page table allocation during which time the page table has not yet been modified.

The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively operates upon vmalloc ranges, and on x86 only within the module mapping space.

This is important, because some callers directly invoke __change_page_attr_set_clr(), bypassing this lock. However, none of these operate within the module mapping space.

* cpa_process_alias() - a recursive helper called by __change_page_attr_set_clr().
* __set_memory_enc_pgtable() - operates on the direct mapping and (via __vmbus_establish_gpadl()) the vmalloc mapping space.
* __set_pages_[n]p() - called by set_direct_map_[invalid, default, valid]_noflush(), __kernel_map_pages() - operates on the direct map.
* kernel_[un]map_pages_in_pgd() - operates on EFI ranges.

This work is based upon Denis V. Lunev's excellent analysis of the bug with gratitude.

---
  arch/x86/mm/pat/set_memory.c | 28 +++++++++++++++++++++++++---
  1 file changed, 25 insertions(+), 3 deletions(-)
arch/x86/mm/pat/set_memory.c +6 -2
index c38faf3..cb5d6d6 100644
@@ -22,6 +22,7 @@
  #include <linux/cc_platform.h>
  #include <linux/set_memory.h>
  #include <linux/memregion.h>
+#include <linux/cleanup.h>
 
  #include <asm/e820/api.h>
  #include <asm/processor.h>
@@ -49,7 +50,8 @@ struct cpa_data {
  	unsigned int	flags;
  	unsigned int	force_split		: 1,
  			force_static_prot	: 1,
-			force_flush_all		: 1;
+			force_flush_all		: 1,
+			init_mm_read_locked	: 1;
  	struct page	**pages;
  };
 
@@ -409,7 +411,7 @@ static void __cpa_flush_tlb(void *data)
 
  static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
 
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
  {
  	unsigned long start, addr, end;
  	struct ptdesc *ptdesc, *tmp;
@@ -443,6 +445,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
  	}
  }
 
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{

arch/x86/mm/pat/set_memory.c static void cpa_collapse_large_pages(struct cpa_data *cpa) · line 459

Quoted 10 lines
arch/x86/mm/pat/set_memory.c +10 -0
+	/*
+	 * Take the mmap write lock on init_mm to:
+	 * - Avoid a use-after-free if raced by ptdump (which takes its own
+	 *   write lock on init_mm).
+	 * - Serialise concurrent CPA walkers.
+	 */
+	scoped_guard(mmap_write_lock, &init_mm)
+		__cpa_collapse_large_pages(cpa);
+}
+

Ah, this belongs to the "Acquire init_mm *write* lock" counterpart. Not sure how you managed to split/combine these two 8-).

thanks,

Signature · 2 lines
js
suse labs
#23 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

Dave - Again this patch has been modified vs. what was submitted and the commit message, tags, etc. are now completely incorrect.

I'm not happy with this going to Linus as-is, is there a way for you to correct these please?

Quoted 136 lines · Jiri Slaby
Quoted 135 lines · On 01. 09. 26, 0:27, tip-bot2 for Lorenzo Stoakes (ARM) wrote:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     477cf5cd1f698053df7426b7b8d9339e85e00946
Gitweb:        https://git.kernel.org/tip/477cf5cd1f698053df7426b7b8d9339e85e00946
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:25 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:15:42 -07:00

x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

A previous commit protected against races between ptdump and CPA collapse, however one still exists between attribute changes and collapse as reported by Denis V. Lunev (linked).

When an attribute change arises, a lockless page table walker obtains a PTE entry, which is later written to via set_pte_atomic():

...
-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()

	-> _lookup_address_cpa()
	-> lookup_address_in_pgd_attr()
	-> [ lockless page table walker ]

-> set_pte_atomic()

There is nothing preventing a concurrent CPA collapse which can free the PTE that was retrieved here, resulting in a use-after-free.

With the mmap write lock taken on init_mm over CPA collapse, resolve this race by acquiring an mmap read lock on init_mm over __change_page_attr_set_clr().

This locks across the whole operation over which the walk and the PTE entry write occurs, solving the race.

It is safe to do this here, as no spinlocks are held upon entry to __change_page_attr_set_clr().

However, the lock must not be held over an allocation, as allocation can trigger reclaim and shrinkers may call into CPA recursively, making deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).

A page table is allocated when a huge page needs to be split:

-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
-> split_large_page()
[ pagetable_alloc() ]
-> __split_large_page()

Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in split_large_page() and track whether this is needed by adding a new 'init_mm_read_locked' flag to struct cpa_data.

This is safe as __split_large_page() (called with locks re-established) revalidates that the page table entry is the same as it was prior to the locks being dropped and __change_page_attr() repeats the entire page table walk whenever a split occurs, so concurrent split and collapse are accounted for.

Concurrent ptdump is also safe as the lock is only dropped over page table allocation during which time the page table has not yet been modified.

The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively operates upon vmalloc ranges, and on x86 only within the module mapping space.

This is important, because some callers directly invoke __change_page_attr_set_clr(), bypassing this lock. However, none of these operate within the module mapping space.

* cpa_process_alias() - a recursive helper called by __change_page_attr_set_clr().
* __set_memory_enc_pgtable() - operates on the direct mapping and (via __vmbus_establish_gpadl()) the vmalloc mapping space.
* __set_pages_[n]p() - called by set_direct_map_[invalid, default, valid]_noflush(), __kernel_map_pages() - operates on the direct map.
* kernel_[un]map_pages_in_pgd() - operates on EFI ranges.

This work is based upon Denis V. Lunev's excellent analysis of the bug with gratitude.

---
  arch/x86/mm/pat/set_memory.c | 28 +++++++++++++++++++++++++---
  1 file changed, 25 insertions(+), 3 deletions(-)
arch/x86/mm/pat/set_memory.c +11 -2
index c38faf3..cb5d6d6 100644
@@ -22,6 +22,7 @@
  #include <linux/cc_platform.h>
  #include <linux/set_memory.h>
  #include <linux/memregion.h>
+#include <linux/cleanup.h>
  #include <asm/e820/api.h>
  #include <asm/processor.h>
@@ -49,7 +50,8 @@ struct cpa_data {
  	unsigned int	flags;
  	unsigned int	force_split		: 1,
  			force_static_prot	: 1,
-			force_flush_all		: 1;
+			force_flush_all		: 1,
+			init_mm_read_locked	: 1;
  	struct page	**pages;
  };
@@ -409,7 +411,7 @@ static void __cpa_flush_tlb(void *data)
  static int collapse_large_pages(unsigned long addr, struct list_head *pgtables);
-static void cpa_collapse_large_pages(struct cpa_data *cpa)
+static void __cpa_collapse_large_pages(struct cpa_data *cpa)
  {
  	unsigned long start, addr, end;
  	struct ptdesc *ptdesc, *tmp;
@@ -443,6 +445,18 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa)
  	}
  }
+static void cpa_collapse_large_pages(struct cpa_data *cpa)
+{
+	/*
+	 * Take the mmap write lock on init_mm to:
+	 * - Avoid a use-after-free if raced by ptdump (which takes its own
+	 *   write lock on init_mm).
+	 * - Serialise concurrent CPA walkers.
Quoted 8 lines

arch/x86/mm/pat/set_memory.c static void cpa_collapse_large_pages(struct cpa_data *cpa) · line 459

Quoted 5 lines
arch/x86/mm/pat/set_memory.c +5 -0
+	 */
+	scoped_guard(mmap_write_lock, &init_mm)
+		__cpa_collapse_large_pages(cpa);
+}
+

Ah, this belongs to the "Acquire init_mm *write* lock" counterpart. Not sure how you managed to split/combine these two 8-).

Yeah this is completely incorrect, now the commit message, tags, etc. are all wrong.

Quoted 6 lines

thanks,
--
js
suse labs

--
Cheers, Lorenzo

#24 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

In reply to Jiri Slaby
Quoted 2 lines · Jiri Slaby

Ah, this belongs to the "Acquire init_mm *write* lock" counterpart. Not sure how you managed to split/combine these two 😎.

Hurrying at the end of the day and being a bonehead will do it. I'm going to back them out from x86/urgent and take a fresh go at it today, post-coffee.

#25 lore raw

[tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     884801a901bd340c44a202e2ef5c29b48e3a4d93
Gitweb:        https://git.kernel.org/tip/884801a901bd340c44a202e2ef5c29b48e3a4d93
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:25 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Tue, 01 Sep 2026 14:35:22 -07:00

x86/mm/pat: Acquire init_mm read lock on attribute change to avoid UAF

A previous commit protected against races between ptdump and CPA collapse, however one still exists between attribute changes and collapse as reported by Denis V. Lunev (linked).

When an attribute change arises, a lockless page table walker obtains a PTE entry, which is later written to via set_pte_atomic():

...
-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()

	-> _lookup_address_cpa()
	-> lookup_address_in_pgd_attr()
	-> [ lockless page table walker ]

-> set_pte_atomic()

There is nothing preventing a concurrent CPA collapse which can free the PTE that was retrieved here, resulting in a use-after-free.

With the mmap write lock taken on init_mm over CPA collapse, resolve this race by acquiring an mmap read lock on init_mm over __change_page_attr_set_clr().

This locks across the whole operation over which the walk and the PTE entry write occurs, solving the race.

It is safe to do this here, as no spinlocks are held upon entry to __change_page_attr_set_clr().

However, the lock must not be held over an allocation, as allocation can trigger reclaim and shrinkers may call into CPA recursively, making deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).

A page table is allocated when a huge page needs to be split:

-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
-> split_large_page()
[ pagetable_alloc() ]
-> __split_large_page()

Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in split_large_page() and track whether this is needed by adding a new 'init_mm_read_locked' flag to struct cpa_data.

This is safe as __split_large_page() (called with locks re-established) revalidates that the page table entry is the same as it was prior to the locks being dropped and __change_page_attr() repeats the entire page table walk whenever a split occurs, so concurrent split and collapse are accounted for.

Concurrent ptdump is also safe as the lock is only dropped over page table allocation during which time the page table has not yet been modified.

The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively operates upon vmalloc ranges, and on x86 only within the module mapping space.

This is important, because some callers directly invoke __change_page_attr_set_clr(), bypassing this lock. However, none of these operate within the module mapping space.

* cpa_process_alias() - a recursive helper called by __change_page_attr_set_clr().
* __set_memory_enc_pgtable() - operates on the direct mapping and (via __vmbus_establish_gpadl()) the vmalloc mapping space.
* __set_pages_[n]p() - called by set_direct_map_[invalid, default, valid]_noflush(), __kernel_map_pages() - operates on the direct map.
* kernel_[un]map_pages_in_pgd() - operates on EFI ranges.

This work is based upon Denis V. Lunev's excellent analysis of the bug with gratitude.

[ dhansen: move to imperative voice in changelog ]

---
 arch/x86/mm/pat/set_memory.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
arch/x86/mm/pat/set_memory.c +11 -2
index 4abddd7..cb5d6d6 100644
@@ -50,7 +50,8 @@ struct cpa_data {
 	unsigned int	flags;
 	unsigned int	force_split		: 1,
 			force_static_prot	: 1,
-			force_flush_all		: 1;
+			force_flush_all		: 1,
+			init_mm_read_locked	: 1;
 	struct page	**pages;
 };
 
@@ -1240,7 +1241,11 @@ static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 	struct ptdesc *ptdesc;
 
 	spin_unlock(&cpa_lock);
+	if (cpa->init_mm_read_locked)
+		mmap_read_unlock(&init_mm);
 	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	if (cpa->init_mm_read_locked)
+		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
 	if (!ptdesc)
 		return -ENOMEM;
@@ -2134,7 +2139,11 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,
 	cpa.curpage = 0;
 	cpa.force_split = force_split;
 
-	ret = __change_page_attr_set_clr(&cpa, 1);
+	/* Avoid race with concurrent CPA collapse. */
+	cpa.init_mm_read_locked = true;
+	scoped_guard(mmap_read_lock, &init_mm)
+		ret = __change_page_attr_set_clr(&cpa, 1);
+	cpa.init_mm_read_locked = false;
 
 	/*
 	 * Check whether we really changed something:
#26 lore raw

[tip: x86/mm] x86/mm/pat: Acquire init_mm read lock on attribute changes to avoid UAF

In reply to Mike Rapoport
The following commit has been merged into the x86/mm branch of tip:

The following commit has been merged into the x86/mm branch of tip:

Commit-ID:     53d22a66b9c675b6dc974955edb9bd6642f3d610
Gitweb:        https://git.kernel.org/tip/53d22a66b9c675b6dc974955edb9bd6642f3d610
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:25 +03:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Tue, 08 Sep 2026 09:07:11 +02:00

x86/mm/pat: Acquire init_mm read lock on attribute changes to avoid UAF

A previous commit protected against races between ptdump and CPA collapse, however one still exists between attribute changes and collapse as reported by Denis V. Lunev (linked).

When an attribute change arises, a lockless page table walker obtains a PTE entry, which is later written to via set_pte_atomic():

...
-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()

	-> _lookup_address_cpa()
	-> lookup_address_in_pgd_attr()
	-> [ lockless page table walker ]

-> set_pte_atomic()

There is nothing preventing a concurrent CPA collapse which can free the PTE that was retrieved here, resulting in a use-after-free.

With the mmap write lock taken on init_mm over CPA collapse, resolve this race by acquiring an mmap read lock on init_mm over __change_page_attr_set_clr().

This locks across the whole operation over which the walk and the PTE entry write occurs, solving the race.

It is safe to do this here, as no spinlocks are held upon entry to __change_page_attr_set_clr().

However, the lock must not be held over an allocation, as allocation can trigger reclaim and shrinkers may call into CPA recursively, making deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).

A page table is allocated when a huge page needs to be split:

-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
-> split_large_page()
[ pagetable_alloc() ]
-> __split_large_page()

Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in split_large_page() and track whether this is needed by adding a new 'init_mm_read_locked' flag to struct cpa_data.

This is safe as __split_large_page() (called with locks re-established) revalidates that the page table entry is the same as it was prior to the locks being dropped and __change_page_attr() repeats the entire page table walk whenever a split occurs, so concurrent split and collapse are accounted for.

Concurrent ptdump is also safe as the lock is only dropped over page table allocation during which time the page table has not yet been modified.

The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively operates upon vmalloc ranges, and on x86 only within the module mapping space.

This is important, because some callers directly invoke __change_page_attr_set_clr(), bypassing this lock. However, none of these operate within the module mapping space.

* cpa_process_alias() - a recursive helper called by

    __change_page_attr_set_clr().

* __set_memory_enc_pgtable() - operates on the direct mapping and (via __vmbus_establish_gpadl()) the vmalloc mapping space.
* __set_pages_[n]p() - called by set_direct_map_[invalid, default, valid]_noflush(), __kernel_map_pages() - operates on the direct map.
* kernel_[un]map_pages_in_pgd() - operates on EFI ranges.

This work is based upon Denis V. Lunev's excellent analysis of the bug with gratitude.

[ dhansen: move to imperative voice in changelog ]

---
 arch/x86/mm/pat/set_memory.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
arch/x86/mm/pat/set_memory.c +11 -2
index 4abddd7..cb5d6d6 100644
@@ -50,7 +50,8 @@ struct cpa_data {
 	unsigned int	flags;
 	unsigned int	force_split		: 1,
 			force_static_prot	: 1,
-			force_flush_all		: 1;
+			force_flush_all		: 1,
+			init_mm_read_locked	: 1;
 	struct page	**pages;
 };
 
@@ -1240,7 +1241,11 @@ static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 	struct ptdesc *ptdesc;
 
 	spin_unlock(&cpa_lock);
+	if (cpa->init_mm_read_locked)
+		mmap_read_unlock(&init_mm);
 	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	if (cpa->init_mm_read_locked)
+		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
 	if (!ptdesc)
 		return -ENOMEM;
@@ -2134,7 +2139,11 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,
 	cpa.curpage = 0;
 	cpa.force_split = force_split;
 
-	ret = __change_page_attr_set_clr(&cpa, 1);
+	/* Avoid race with concurrent CPA collapse. */
+	cpa.init_mm_read_locked = true;
+	scoped_guard(mmap_read_lock, &init_mm)
+		ret = __change_page_attr_set_clr(&cpa, 1);
+	cpa.init_mm_read_locked = false;
 
 	/*
 	 * Check whether we really changed something:
#27 lore raw

[tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute changes to avoid UAF

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     ac20c594d88b2809b7d6a6e496ab16e0a39b82a9
Gitweb:        https://git.kernel.org/tip/ac20c594d88b2809b7d6a6e496ab16e0a39b82a9
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:25 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Tue, 08 Sep 2026 15:43:06 -07:00

x86/mm/pat: Acquire init_mm read lock on attribute changes to avoid UAF

A previous commit protected against races between ptdump and CPA collapse, however one still exists between attribute changes and collapse as reported by Denis V. Lunev (linked).

When an attribute change arises, a lockless page table walker obtains a PTE entry, which is later written to via set_pte_atomic():

...
-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()

	-> _lookup_address_cpa()
	-> lookup_address_in_pgd_attr()
	-> [ lockless page table walker ]

-> set_pte_atomic()

There is nothing preventing a concurrent CPA collapse which can free the PTE that was retrieved here, resulting in a use-after-free.

With the mmap write lock taken on init_mm over CPA collapse, resolve this race by acquiring an mmap read lock on init_mm over __change_page_attr_set_clr().

This locks across the whole operation over which the walk and the PTE entry write occurs, solving the race.

It is safe to do this here, as no spinlocks are held upon entry to __change_page_attr_set_clr().

However, the lock must not be held over an allocation, as allocation can trigger reclaim and shrinkers may call into CPA recursively, making deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).

A page table is allocated when a huge page needs to be split:

-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
-> split_large_page()
[ pagetable_alloc() ]
-> __split_large_page()

Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in split_large_page() and track whether this is needed by adding a new 'init_mm_read_locked' flag to struct cpa_data.

This is safe as __split_large_page() (called with locks re-established) revalidates that the page table entry is the same as it was prior to the locks being dropped and __change_page_attr() repeats the entire page table walk whenever a split occurs, so concurrent split and collapse are accounted for.

Concurrent ptdump is also safe as the lock is only dropped over page table allocation during which time the page table has not yet been modified.

The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively operates upon vmalloc ranges, and on x86 only within the module mapping space.

This is important, because some callers directly invoke __change_page_attr_set_clr(), bypassing this lock. However, none of these operate within the module mapping space.

* cpa_process_alias() - a recursive helper called by

    __change_page_attr_set_clr().

* __set_memory_enc_pgtable() - operates on the direct mapping and (via __vmbus_establish_gpadl()) the vmalloc mapping space.
* __set_pages_[n]p() - called by set_direct_map_[invalid, default, valid]_noflush(), __kernel_map_pages() - operates on the direct map.
* kernel_[un]map_pages_in_pgd() - operates on EFI ranges.

This work is based upon Denis V. Lunev's excellent analysis of the bug with gratitude.

[ dhansen: move to imperative voice in changelog ]

---
 arch/x86/mm/pat/set_memory.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
arch/x86/mm/pat/set_memory.c +11 -2
index 4abddd7..cb5d6d6 100644
@@ -50,7 +50,8 @@ struct cpa_data {
 	unsigned int	flags;
 	unsigned int	force_split		: 1,
 			force_static_prot	: 1,
-			force_flush_all		: 1;
+			force_flush_all		: 1,
+			init_mm_read_locked	: 1;
 	struct page	**pages;
 };
 
@@ -1240,7 +1241,11 @@ static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 	struct ptdesc *ptdesc;
 
 	spin_unlock(&cpa_lock);
+	if (cpa->init_mm_read_locked)
+		mmap_read_unlock(&init_mm);
 	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	if (cpa->init_mm_read_locked)
+		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
 	if (!ptdesc)
 		return -ENOMEM;
@@ -2134,7 +2139,11 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,
 	cpa.curpage = 0;
 	cpa.force_split = force_split;
 
-	ret = __change_page_attr_set_clr(&cpa, 1);
+	/* Avoid race with concurrent CPA collapse. */
+	cpa.init_mm_read_locked = true;
+	scoped_guard(mmap_read_lock, &init_mm)
+		ret = __change_page_attr_set_clr(&cpa, 1);
+	cpa.init_mm_read_locked = false;
 
 	/*
 	 * Check whether we really changed something:
#28 lore raw

[tip: x86/urgent] x86/mm/pat: Acquire init_mm read lock on attribute changes to avoid UAF

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     d5d8b8662e6e5a565b47a0388640e88402f23274
Gitweb:        https://git.kernel.org/tip/d5d8b8662e6e5a565b47a0388640e88402f23274
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:25 +03:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 09 Sep 2026 08:39:04 +02:00

x86/mm/pat: Acquire init_mm read lock on attribute changes to avoid UAF

A previous commit protected against races between ptdump and CPA collapse, however one still exists between attribute changes and collapse as reported by Denis V. Lunev (linked).

When an attribute change arises, a lockless page table walker obtains a PTE entry, which is later written to via set_pte_atomic():

...
-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()

	-> _lookup_address_cpa()
	-> lookup_address_in_pgd_attr()
	-> [ lockless page table walker ]

-> set_pte_atomic()

There is nothing preventing a concurrent CPA collapse which can free the PTE that was retrieved here, resulting in a use-after-free.

With the mmap write lock taken on init_mm over CPA collapse, resolve this race by acquiring an mmap read lock on init_mm over __change_page_attr_set_clr().

This locks across the whole operation over which the walk and the PTE entry write occurs, solving the race.

It is safe to do this here, as no spinlocks are held upon entry to __change_page_attr_set_clr().

However, the lock must not be held over an allocation, as allocation can trigger reclaim and shrinkers may call into CPA recursively, making deadlocks possible (init_mm -> ... -> fs_reclaim -> init_mm).

A page table is allocated when a huge page needs to be split:

-> change_page_attr_set_clr()
-> __change_page_attr_set_clr()
-> __change_page_attr()
-> split_large_page()
[ pagetable_alloc() ]
-> __split_large_page()

Avoid deadlocks by dropping the mmap lock across pagetable_alloc() in split_large_page() and track whether this is needed by adding a new 'init_mm_read_locked' flag to struct cpa_data.

This is safe as __split_large_page() (called with locks re-established) revalidates that the page table entry is the same as it was prior to the locks being dropped and __change_page_attr() repeats the entire page table walk whenever a split occurs, so concurrent split and collapse are accounted for.

Concurrent ptdump is also safe as the lock is only dropped over page table allocation during which time the page table has not yet been modified.

The CPA_COLLAPSE flag is only set by set_memory_rox(), which exclusively operates upon vmalloc ranges, and on x86 only within the module mapping space.

This is important, because some callers directly invoke __change_page_attr_set_clr(), bypassing this lock. However, none of these operate within the module mapping space.

* cpa_process_alias() - a recursive helper called by

    __change_page_attr_set_clr().

* __set_memory_enc_pgtable() - operates on the direct mapping and (via __vmbus_establish_gpadl()) the vmalloc mapping space.
* __set_pages_[n]p() - called by set_direct_map_[invalid, default, valid]_noflush(), __kernel_map_pages() - operates on the direct map.
* kernel_[un]map_pages_in_pgd() - operates on EFI ranges.

This work is based upon Denis V. Lunev's excellent analysis of the bug with gratitude.

[ dhansen: move to imperative voice in changelog ]

---
 arch/x86/mm/pat/set_memory.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
arch/x86/mm/pat/set_memory.c +11 -2
index 4abddd7..cb5d6d6 100644
@@ -50,7 +50,8 @@ struct cpa_data {
 	unsigned int	flags;
 	unsigned int	force_split		: 1,
 			force_static_prot	: 1,
-			force_flush_all		: 1;
+			force_flush_all		: 1,
+			init_mm_read_locked	: 1;
 	struct page	**pages;
 };
 
@@ -1240,7 +1241,11 @@ static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 	struct ptdesc *ptdesc;
 
 	spin_unlock(&cpa_lock);
+	if (cpa->init_mm_read_locked)
+		mmap_read_unlock(&init_mm);
 	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	if (cpa->init_mm_read_locked)
+		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
 	if (!ptdesc)
 		return -ENOMEM;
@@ -2134,7 +2139,11 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,
 	cpa.curpage = 0;
 	cpa.force_split = force_split;
 
-	ret = __change_page_attr_set_clr(&cpa, 1);
+	/* Avoid race with concurrent CPA collapse. */
+	cpa.init_mm_read_locked = true;
+	scoped_guard(mmap_read_lock, &init_mm)
+		ret = __change_page_attr_set_clr(&cpa, 1);
+	cpa.init_mm_read_locked = false;
 
 	/*
 	 * Check whether we really changed something:
#29 lore raw

PATCH v2 3/5 x86/alternative: exclude text poking against change_page_attr()

In reply to Mike Rapoport

From: Pedro Falcato <[email protected]>

From time to time, the following BUG can be observed[0]:

Quoted 12 lines

kernel BUG at arch/x86/kernel/alternative.c:2576!

Oops: invalid opcode: 0000 [#1] SMP NOPTI
CPU: 0 UID: 0 PID: 355 Comm: (udev-worker) Not tainted 7.1.3-1-default #1 PREEMPT(full) openSUSE Tumbleweed  8c1795b03ec64f997e57a8ad38b1161e3b98da64
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS unknown 02/02/2022
RIP: 0010:__text_poke+0x2aa/0x450
Call Trace:
 <TASK>
 smp_text_poke_batch_finish+0x2a7/0x320
 __static_call_transform+0xb7/0x220
 arch_static_call_transform+0x5b/0xb0
 __static_call_init+0xe9/0x270
 static_call_module_notify+0x11f/0x150
Quoted 8 lines
 notifier_call_chain+0x61/0xe0
 blocking_notifier_call_chain_robust+0x63/0xc0
 load_module+0x1c92/0x20c0
 init_module_from_file+0xd8/0x140
 idempotent_init_module+0x100/0x2f0
 __x64_sys_finit_module+0x71/0xe0
 do_syscall_64+0xe1/0x610
 entry_SYSCALL_64_after_hwframe+0x76/0x7e

which matches the following BUG_ON in alternative.c:

	/*
	 * If something went wrong, crash and burn since recovery paths are not
	 * implemented.
	 */
	BUG_ON(!pages[0] || (cross_page_boundary && !pages[1]));

This can happen if vmalloc_to_page() fails, for any reason. Such can happen if text poking races with CPA, which can possibly result in the collapsing of page tables (or breaking of PMD hugepages). It is not a problem for most users of vmalloc_to_page() (they solely own the vmalloc'd range) but, when CONFIG_ARCH_HAS_EXECMEM_ROX=y, various modules own a single execmem vmalloc range, and can call set_memory_*() in parallel on it. This can happen to race against __text_poke and cause havoc in vmalloc_to_page().

Fix it by excluding against CPA using the init_mm mmap read lock.

---
 arch/x86/kernel/alternative.c | 39 ++++++++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)
arch/x86/kernel/alternative.c +36 -3
index 62936a3bde19..f81d6bc90a6f 100644
@@ -6,6 +6,9 @@
 #include <linux/vmalloc.h>
 #include <linux/memory.h>
 #include <linux/execmem.h>
+#include <linux/cleanup.h>
+#include <linux/kgdb.h>
+#include <linux/mmap_lock.h>
 
 #include <asm/text-patching.h>
 #include <asm/insn.h>
@@ -2543,6 +2546,38 @@ static void text_poke_memset(void *dst, const void *src, size_t len)
 
 typedef void text_poke_f(void *dst, const void *src, size_t len);
 
+static void __poke_vmalloc_pages(struct page **pages, void *addr,
+				 bool cross_page_boundary)
+{
+	pages[0] = vmalloc_to_page(addr);
+	if (cross_page_boundary)
+		pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+}
+
+static void poke_vmalloc_pages(struct page **pages, void *addr,
+			       bool cross_page_boundary)
+{
+	if (in_dbg_master()) {
+		/*
+		 * If called from kgdb cannot sleep, but all other CPUs stopped
+		 * anyway so safe to proceed without locks
+		 */
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	} else {
+		/*
+		 * execmem ROX ranges are shared between modules and can be
+		 * collapsed to huge PMD entries, and this collapse can happen
+		 * concurrently with a racing set_memory_rox().
+		 *
+		 * Prevent vmalloc_to_page() from racing by acquiring an
+		 * init_mm read lock which pairs with the init_mm write lock in
+		 * cpa_collapse_large_pages().
+		 */
+		guard(mmap_read_lock)(&init_mm);
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	}
+}
+
 static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t len)
 {
 	bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
@@ -2560,9 +2595,7 @@ static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l
 	BUG_ON(!after_bootmem);
 
 	if (!core_kernel_text((unsigned long)addr)) {
-		pages[0] = vmalloc_to_page(addr);
-		if (cross_page_boundary)
-			pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+		poke_vmalloc_pages(pages, addr, cross_page_boundary);
 	} else {
 		pages[0] = virt_to_page(addr);
 		WARN_ON(!PageReserved(pages[0]));
Signature · 1 line
2.53.0
Tested-by on v2 3/5
#30 lore raw

Re: [PATCH v2 3/5] x86/alternative: exclude text poking against change_page_attr()

Quoted 40 lines · Mike Rapoport

From: Pedro Falcato <[email protected]>

From time to time, the following BUG can be observed[0]:

Quoted 12 lines

kernel BUG at arch/x86/kernel/alternative.c:2576!

Oops: invalid opcode: 0000 [#1] SMP NOPTI
CPU: 0 UID: 0 PID: 355 Comm: (udev-worker) Not tainted 7.1.3-1-default #1 PREEMPT(full) openSUSE Tumbleweed  8c1795b03ec64f997e57a8ad38b1161e3b98da64
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS unknown 02/02/2022
RIP: 0010:__text_poke+0x2aa/0x450
Call Trace:
  <TASK>
  smp_text_poke_batch_finish+0x2a7/0x320
  __static_call_transform+0xb7/0x220
  arch_static_call_transform+0x5b/0xb0
  __static_call_init+0xe9/0x270
  static_call_module_notify+0x11f/0x150
Quoted 8 lines
  notifier_call_chain+0x61/0xe0
  blocking_notifier_call_chain_robust+0x63/0xc0
  load_module+0x1c92/0x20c0
  init_module_from_file+0xd8/0x140
  idempotent_init_module+0x100/0x2f0
  __x64_sys_finit_module+0x71/0xe0
  do_syscall_64+0xe1/0x610
  entry_SYSCALL_64_after_hwframe+0x76/0x7e

which matches the following BUG_ON in alternative.c:

	/*
	 * If something went wrong, crash and burn since recovery paths are not
	 * implemented.
	 */
	BUG_ON(!pages[0] || (cross_page_boundary && !pages[1]));

This can happen if vmalloc_to_page() fails, for any reason. Such can happen if text poking races with CPA, which can possibly result in the collapsing of page tables (or breaking of PMD hugepages). It is not a problem for most users of vmalloc_to_page() (they solely own the vmalloc'd range) but, when CONFIG_ARCH_HAS_EXECMEM_ROX=y, various modules own a single execmem vmalloc range, and can call set_memory_*() in parallel on it. This can happen to race against __text_poke and cause havoc in vmalloc_to_page().

Quoted 4 lines

Fix it by excluding against CPA using the init_mm mmap read lock.

Fixes 64f6a4e10c05 ("x86: re-enable EXECMEM_ROX support")

FWIW

We have not seen any BUGs since applied to the SUSE's kernel.
https://bugzilla.suse.com/show_bug.cgi?id=1271202#c26

thanks,

Signature · 2 lines
js
suse labs
merge bot Applied
#31 lore raw

[tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     1c4d1c070d734b241728378c0f79950c23a10883
Gitweb:        https://git.kernel.org/tip/1c4d1c070d734b241728378c0f79950c23a10883
Author:        Pedro Falcato <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:17:35 -07:00

x86/alternative: Exclude text poking against change_page_attr()

Quoted 14 lines

From time to time, the following BUG can be observed[0]:

kernel BUG at arch/x86/kernel/alternative.c:2576!

Oops: invalid opcode: 0000 [#1] SMP NOPTI
CPU: 0 UID: 0 PID: 355 Comm: (udev-worker) Not tainted 7.1.3-1-default #1 PREEMPT(full) openSUSE Tumbleweed  8c1795b03ec64f997e57a8ad38b1161e3b98da64
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS unknown 02/02/2022
RIP: 0010:__text_poke+0x2aa/0x450
Call Trace:
 <TASK>
 smp_text_poke_batch_finish+0x2a7/0x320
 __static_call_transform+0xb7/0x220
 arch_static_call_transform+0x5b/0xb0
 __static_call_init+0xe9/0x270
 static_call_module_notify+0x11f/0x150
Quoted 8 lines
 notifier_call_chain+0x61/0xe0
 blocking_notifier_call_chain_robust+0x63/0xc0
 load_module+0x1c92/0x20c0
 init_module_from_file+0xd8/0x140
 idempotent_init_module+0x100/0x2f0
 __x64_sys_finit_module+0x71/0xe0
 do_syscall_64+0xe1/0x610
 entry_SYSCALL_64_after_hwframe+0x76/0x7e

which matches the following BUG_ON in alternative.c:

	/*
	 * If something went wrong, crash and burn since recovery paths are not
	 * implemented.
	 */
	BUG_ON(!pages[0] || (cross_page_boundary && !pages[1]));

This can happen if vmalloc_to_page() fails, for any reason. Such can happen if text poking races with CPA, which can possibly result in the collapsing of page tables (or breaking of PMD hugepages). It is not a problem for most users of vmalloc_to_page() (they solely own the vmalloc'd range) but, when CONFIG_ARCH_HAS_EXECMEM_ROX=y, various modules own a single execmem vmalloc range, and can call set_memory_*() in parallel on it. This can happen to race against __text_poke and cause havoc in vmalloc_to_page().

Fix it by excluding against CPA using the init_mm mmap read lock.

---
 arch/x86/kernel/alternative.c | 39 +++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)
arch/x86/kernel/alternative.c +36 -3
index 91b1cdd..add62db 100644
@@ -6,6 +6,9 @@
 #include <linux/vmalloc.h>
 #include <linux/memory.h>
 #include <linux/execmem.h>
+#include <linux/cleanup.h>
+#include <linux/kgdb.h>
+#include <linux/mmap_lock.h>
 
 #include <asm/text-patching.h>
 #include <asm/insn.h>
@@ -2372,6 +2375,38 @@ static void text_poke_memset(void *dst, const void *src, size_t len)
 
 typedef void text_poke_f(void *dst, const void *src, size_t len);
 
+static void __poke_vmalloc_pages(struct page **pages, void *addr,
+				 bool cross_page_boundary)
+{
+	pages[0] = vmalloc_to_page(addr);
+	if (cross_page_boundary)
+		pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+}
+
+static void poke_vmalloc_pages(struct page **pages, void *addr,
+			       bool cross_page_boundary)
+{
+	if (in_dbg_master()) {
+		/*
+		 * If called from kgdb cannot sleep, but all other CPUs stopped
+		 * anyway so safe to proceed without locks
+		 */
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	} else {
+		/*
+		 * execmem ROX ranges are shared between modules and can be
+		 * collapsed to huge PMD entries, and this collapse can happen
+		 * concurrently with a racing set_memory_rox().
+		 *
+		 * Prevent vmalloc_to_page() from racing by acquiring an
+		 * init_mm read lock which pairs with the init_mm write lock in
+		 * cpa_collapse_large_pages().
+		 */
+		guard(mmap_read_lock)(&init_mm);
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	}
+}
+
 static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t len)
 {
 	bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
@@ -2389,9 +2424,7 @@ static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l
 	BUG_ON(!after_bootmem);
 
 	if (!core_kernel_text((unsigned long)addr)) {
-		pages[0] = vmalloc_to_page(addr);
-		if (cross_page_boundary)
-			pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+		poke_vmalloc_pages(pages, addr, cross_page_boundary);
 	} else {
 		pages[0] = virt_to_page(addr);
 		WARN_ON(!PageReserved(pages[0]));
#32 lore raw

Re: [tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

Quoted 4 lines · On 01. 09. 26, 0:27, tip-bot2 for Pedro Falcato wrote:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     1c4d1c070d734b241728378c0f79950c23a10883
Gitweb:        https://git.kernel.org/tip/1c4d1c070d734b241728378c0f79950c23a10883

Was it dropped in the meantime? Gitweb says:
Object 1c4d1c070d734b241728378c0f79950c23a10883 is not reachable from any reference in this repository

$ git rev-parse tip/x86/urgent
477cf5cd1f698053df7426b7b8d9339e85e00946
$ git branch -r --contains 1c4d1c070d734b241728378c0f79950c23a10883
$

Despite I managed to download it at some point though:

$ git show --oneline -s 1c4d1c070d734b241728378c0f79950c23a10883
1c4d1c070d73 x86/alternative: Exclude text poking against change_page_attr()

What's going on with tip today?

Quoted 6 lines
Author:        Pedro Falcato <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:17:35 -07:00

x86/alternative: Exclude text poking against change_page_attr()

thanks,

Signature · 2 lines
js
suse labs
#33 lore raw

Re: [tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

Quoted 15 lines · Jiri Slaby
Quoted 4 lines · On 01. 09. 26, 0:27, tip-bot2 for Pedro Falcato wrote:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     1c4d1c070d734b241728378c0f79950c23a10883
Gitweb:        https://git.kernel.org/tip/1c4d1c070d734b241728378c0f79950c23a10883

Was it dropped in the meantime? Gitweb says:
Object 1c4d1c070d734b241728378c0f79950c23a10883 is not reachable from any reference in this repository

$ git rev-parse tip/x86/urgent
477cf5cd1f698053df7426b7b8d9339e85e00946
$ git branch -r --contains 1c4d1c070d734b241728378c0f79950c23a10883
$
Quoted 5 lines

Despite I managed to download it at some point though:

$ git show --oneline -s 1c4d1c070d734b241728378c0f79950c23a10883
1c4d1c070d73 x86/alternative: Exclude text poking against change_page_attr()

What's going on with tip today?

Yup, I can't see this commit _at all_ in x86/urgent. Something's very broken here.

Quoted 11 lines
Quoted 7 lines
Author:        Pedro Falcato <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:17:35 -07:00

x86/alternative: Exclude text poking against change_page_attr()

thanks,
--
js
suse labs

--
Cheers, Lorenzo

#34 lore raw

Re: [tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

Quoted 16 lines · Lorenzo Stoakes (ARM)
Quoted 15 lines · Jiri Slaby
Quoted 4 lines · On 01. 09. 26, 0:27, tip-bot2 for Pedro Falcato wrote:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     1c4d1c070d734b241728378c0f79950c23a10883
Gitweb:        https://git.kernel.org/tip/1c4d1c070d734b241728378c0f79950c23a10883

Was it dropped in the meantime? Gitweb says:
Object 1c4d1c070d734b241728378c0f79950c23a10883 is not reachable from any reference in this repository

$ git rev-parse tip/x86/urgent
477cf5cd1f698053df7426b7b8d9339e85e00946
$ git branch -r --contains 1c4d1c070d734b241728378c0f79950c23a10883
$
Quoted 8 lines
Quoted 5 lines

Despite I managed to download it at some point though:

$ git show --oneline -s 1c4d1c070d734b241728378c0f79950c23a10883
1c4d1c070d73 x86/alternative: Exclude text poking against change_page_attr()

What's going on with tip today?

Yup, I can't see this commit _at all_ in x86/urgent. Something's very broken here.

FTR, I dug out the logs:
+ 096edb090797...4a0e6473921a master -> tip/master (forced update)
+ 096edb090797...4a0e6473921a tip/urgent -> tip/tip/urgent (forced
update)
+ 34cd1c931a36...477cf5cd1f69 x86/urgent -> tip/x86/urgent (forced
update)

The original x86/urgent's HEAD (34cd1c931a36) indeed contains the commit (1c4d1c070d7).

Quoted 13 lines
Quoted 10 lines
Quoted 6 lines
Author:        Pedro Falcato <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:17:35 -07:00

x86/alternative: Exclude text poking against change_page_attr()

thanks,
--
js
suse labs

--
Cheers, Lorenzo

Signature · 2 lines
js
suse labs
#35 lore raw

Re: [tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

Quoted 26 lines · Jiri Slaby
Quoted 16 lines · Lorenzo Stoakes (ARM)
Quoted 15 lines · Jiri Slaby
Quoted 4 lines · On 01. 09. 26, 0:27, tip-bot2 for Pedro Falcato wrote:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     1c4d1c070d734b241728378c0f79950c23a10883
Gitweb:        https://git.kernel.org/tip/1c4d1c070d734b241728378c0f79950c23a10883

Was it dropped in the meantime? Gitweb says:
Object 1c4d1c070d734b241728378c0f79950c23a10883 is not reachable from any reference in this repository

$ git rev-parse tip/x86/urgent
477cf5cd1f698053df7426b7b8d9339e85e00946
$ git branch -r --contains 1c4d1c070d734b241728378c0f79950c23a10883
$
Quoted 8 lines
Quoted 5 lines

Despite I managed to download it at some point though:

$ git show --oneline -s 1c4d1c070d734b241728378c0f79950c23a10883
1c4d1c070d73 x86/alternative: Exclude text poking against change_page_attr()

What's going on with tip today?

Yup, I can't see this commit _at all_ in x86/urgent. Something's very broken here.

Quoted 7 lines

FTR, I dug out the logs:
+ 096edb090797...4a0e6473921a master -> tip/master (forced update)
+ 096edb090797...4a0e6473921a tip/urgent -> tip/tip/urgent (forced update)
+ 34cd1c931a36...477cf5cd1f69 x86/urgent -> tip/x86/urgent (forced update)

The original x86/urgent's HEAD (34cd1c931a36) indeed contains the commit (1c4d1c070d7).

I mean 'forced updated' gives me hope at least that these patches can be rebased back to sanity :)

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Quoted 7 lines
Author:        Pedro Falcato <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:17:35 -07:00

x86/alternative: Exclude text poking against change_page_attr()

thanks,
--
js
suse labs

Quoted 7 lines
Quoted 2 lines

--
Cheers, Lorenzo

--
js
suse labs

--
Cheers, Lorenzo

merge bot Applied
#36 lore raw

[tip: x86/urgent] x86/alternative: Exclude text poking against change_page_attr()

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     e679ba0983757e9567aeda97cc35c99241d420ee
Gitweb:        https://git.kernel.org/tip/e679ba0983757e9567aeda97cc35c99241d420ee
Author:        Pedro Falcato <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Tue, 01 Sep 2026 14:54:28 -07:00

x86/alternative: Exclude text poking against change_page_attr()

Quoted 14 lines

From time to time, the following BUG can be observed[0]:

kernel BUG at arch/x86/kernel/alternative.c:2576!

Oops: invalid opcode: 0000 [#1] SMP NOPTI
CPU: 0 UID: 0 PID: 355 Comm: (udev-worker) Not tainted 7.1.3-1-default #1 PREEMPT(full) openSUSE Tumbleweed  8c1795b03ec64f997e57a8ad38b1161e3b98da64
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS unknown 02/02/2022
RIP: 0010:__text_poke+0x2aa/0x450
Call Trace:
 <TASK>
 smp_text_poke_batch_finish+0x2a7/0x320
 __static_call_transform+0xb7/0x220
 arch_static_call_transform+0x5b/0xb0
 __static_call_init+0xe9/0x270
 static_call_module_notify+0x11f/0x150
Quoted 8 lines
 notifier_call_chain+0x61/0xe0
 blocking_notifier_call_chain_robust+0x63/0xc0
 load_module+0x1c92/0x20c0
 init_module_from_file+0xd8/0x140
 idempotent_init_module+0x100/0x2f0
 __x64_sys_finit_module+0x71/0xe0
 do_syscall_64+0xe1/0x610
 entry_SYSCALL_64_after_hwframe+0x76/0x7e

which matches the following BUG_ON in alternative.c:

	/*
	 * If something went wrong, crash and burn since recovery paths are not
	 * implemented.
	 */
	BUG_ON(!pages[0] || (cross_page_boundary && !pages[1]));

This can happen if vmalloc_to_page() fails, for any reason. Such can happen if text poking races with CPA, which can possibly result in the collapsing of page tables (or breaking of PMD hugepages). It is not a problem for most users of vmalloc_to_page() (they solely own the vmalloc'd range) but, when CONFIG_ARCH_HAS_EXECMEM_ROX=y, various modules own a single execmem vmalloc range, and can call set_memory_*() in parallel on it. This can happen to race against __text_poke and cause havoc in vmalloc_to_page().

Fix it by excluding against CPA using the init_mm mmap read lock.

[ dhansen: Fix up SoB ordering. The actual code flow here was:
Pedro=>Lorenzo=>Mike=>Me which is reflected in the SoB chain now. I *believe* Mike simply picked up Lorenzo's update to Pedro's post from the Link: ]

---
 arch/x86/kernel/alternative.c | 39 +++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)
arch/x86/kernel/alternative.c +36 -3
index 91b1cdd..add62db 100644
@@ -6,6 +6,9 @@
 #include <linux/vmalloc.h>
 #include <linux/memory.h>
 #include <linux/execmem.h>
+#include <linux/cleanup.h>
+#include <linux/kgdb.h>
+#include <linux/mmap_lock.h>
 
 #include <asm/text-patching.h>
 #include <asm/insn.h>
@@ -2372,6 +2375,38 @@ static void text_poke_memset(void *dst, const void *src, size_t len)
 
 typedef void text_poke_f(void *dst, const void *src, size_t len);
 
+static void __poke_vmalloc_pages(struct page **pages, void *addr,
+				 bool cross_page_boundary)
+{
+	pages[0] = vmalloc_to_page(addr);
+	if (cross_page_boundary)
+		pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+}
+
+static void poke_vmalloc_pages(struct page **pages, void *addr,
+			       bool cross_page_boundary)
+{
+	if (in_dbg_master()) {
+		/*
+		 * If called from kgdb cannot sleep, but all other CPUs stopped
+		 * anyway so safe to proceed without locks
+		 */
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	} else {
+		/*
+		 * execmem ROX ranges are shared between modules and can be
+		 * collapsed to huge PMD entries, and this collapse can happen
+		 * concurrently with a racing set_memory_rox().
+		 *
+		 * Prevent vmalloc_to_page() from racing by acquiring an
+		 * init_mm read lock which pairs with the init_mm write lock in
+		 * cpa_collapse_large_pages().
+		 */
+		guard(mmap_read_lock)(&init_mm);
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	}
+}
+
 static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t len)
 {
 	bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
@@ -2389,9 +2424,7 @@ static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l
 	BUG_ON(!after_bootmem);
 
 	if (!core_kernel_text((unsigned long)addr)) {
-		pages[0] = vmalloc_to_page(addr);
-		if (cross_page_boundary)
-			pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+		poke_vmalloc_pages(pages, addr, cross_page_boundary);
 	} else {
 		pages[0] = virt_to_page(addr);
 		WARN_ON(!PageReserved(pages[0]));
merge bot Applied
#37 lore raw

[tip: x86/mm] x86/alternatives: Exclude text poking against change_page_attr()

In reply to Mike Rapoport
The following commit has been merged into the x86/mm branch of tip:

The following commit has been merged into the x86/mm branch of tip:

Commit-ID:     d14648849f1b1616aa91475026991c8bf78519d4
Gitweb:        https://git.kernel.org/tip/d14648849f1b1616aa91475026991c8bf78519d4
Author:        Pedro Falcato <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Tue, 08 Sep 2026 09:10:10 +02:00

x86/alternatives: Exclude text poking against change_page_attr()

Quoted 1 line

From time to time, the following BUG can be observed

in the x86 alternatives patching code [0]:

Quoted 12 lines

kernel BUG at arch/x86/kernel/alternative.c:2576!

Oops: invalid opcode: 0000 [#1] SMP NOPTI
CPU: 0 UID: 0 PID: 355 Comm: (udev-worker) Not tainted 7.1.3-1-default #1 PREEMPT(full) openSUSE Tumbleweed  8c1795b03ec64f997e57a8ad38b1161e3b98da64
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS unknown 02/02/2022
RIP: 0010:__text_poke+0x2aa/0x450
Call Trace:
 <TASK>
 smp_text_poke_batch_finish+0x2a7/0x320
 __static_call_transform+0xb7/0x220
 arch_static_call_transform+0x5b/0xb0
 __static_call_init+0xe9/0x270
 static_call_module_notify+0x11f/0x150
Quoted 8 lines
 notifier_call_chain+0x61/0xe0
 blocking_notifier_call_chain_robust+0x63/0xc0
 load_module+0x1c92/0x20c0
 init_module_from_file+0xd8/0x140
 idempotent_init_module+0x100/0x2f0
 __x64_sys_finit_module+0x71/0xe0
 do_syscall_64+0xe1/0x610
 entry_SYSCALL_64_after_hwframe+0x76/0x7e

which matches the following BUG_ON() in alternative.c:

	/*
	 * If something went wrong, crash and burn since recovery paths are not
	 * implemented.
	 */
	BUG_ON(!pages[0] || (cross_page_boundary && !pages[1]));

This can happen if vmalloc_to_page() fails, for any reason. Such can happen if text poking races with CPA, which can possibly result in the collapsing of page tables (or breaking of PMD hugepages). It is not a problem for most users of vmalloc_to_page() (they solely own the vmalloc'd range) but, when CONFIG_ARCH_HAS_EXECMEM_ROX=y, various modules own a single execmem vmalloc range, and can call set_memory_*() in parallel on it. This can happen to race against __text_poke and cause havoc in vmalloc_to_page().

Fix it by excluding against CPA using the init_mm mmap read lock.

[ dhansen: Fix up SoB ordering. The actual code flow here was:
Pedro=>Lorenzo=>Mike=>Me which is reflected in the SoB chain now. I *believe* Mike simply picked up Lorenzo's update to Pedro's post from the Link ]

---
 arch/x86/kernel/alternative.c | 39 +++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)
arch/x86/kernel/alternative.c +36 -3
index 91b1cdd..add62db 100644
@@ -6,6 +6,9 @@
 #include <linux/vmalloc.h>
 #include <linux/memory.h>
 #include <linux/execmem.h>
+#include <linux/cleanup.h>
+#include <linux/kgdb.h>
+#include <linux/mmap_lock.h>
 
 #include <asm/text-patching.h>
 #include <asm/insn.h>
@@ -2372,6 +2375,38 @@ static void text_poke_memset(void *dst, const void *src, size_t len)
 
 typedef void text_poke_f(void *dst, const void *src, size_t len);
 
+static void __poke_vmalloc_pages(struct page **pages, void *addr,
+				 bool cross_page_boundary)
+{
+	pages[0] = vmalloc_to_page(addr);
+	if (cross_page_boundary)
+		pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+}
+
+static void poke_vmalloc_pages(struct page **pages, void *addr,
+			       bool cross_page_boundary)
+{
+	if (in_dbg_master()) {
+		/*
+		 * If called from kgdb cannot sleep, but all other CPUs stopped
+		 * anyway so safe to proceed without locks
+		 */
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	} else {
+		/*
+		 * execmem ROX ranges are shared between modules and can be
+		 * collapsed to huge PMD entries, and this collapse can happen
+		 * concurrently with a racing set_memory_rox().
+		 *
+		 * Prevent vmalloc_to_page() from racing by acquiring an
+		 * init_mm read lock which pairs with the init_mm write lock in
+		 * cpa_collapse_large_pages().
+		 */
+		guard(mmap_read_lock)(&init_mm);
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	}
+}
+
 static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t len)
 {
 	bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
@@ -2389,9 +2424,7 @@ static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l
 	BUG_ON(!after_bootmem);
 
 	if (!core_kernel_text((unsigned long)addr)) {
-		pages[0] = vmalloc_to_page(addr);
-		if (cross_page_boundary)
-			pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+		poke_vmalloc_pages(pages, addr, cross_page_boundary);
 	} else {
 		pages[0] = virt_to_page(addr);
 		WARN_ON(!PageReserved(pages[0]));
merge bot Applied
#38 lore raw

[tip: x86/urgent] x86/alternatives: Exclude text poking against change_page_attr()

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     68ad13e04081e7bc3c8791fbb5b41f8093167c12
Gitweb:        https://git.kernel.org/tip/68ad13e04081e7bc3c8791fbb5b41f8093167c12
Author:        Pedro Falcato <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Tue, 08 Sep 2026 15:43:07 -07:00

x86/alternatives: Exclude text poking against change_page_attr()

Quoted 1 line

From time to time, the following BUG can be observed

in the x86 alternatives patching code [0]:

Quoted 12 lines

kernel BUG at arch/x86/kernel/alternative.c:2576!

Oops: invalid opcode: 0000 [#1] SMP NOPTI
CPU: 0 UID: 0 PID: 355 Comm: (udev-worker) Not tainted 7.1.3-1-default #1 PREEMPT(full) openSUSE Tumbleweed  8c1795b03ec64f997e57a8ad38b1161e3b98da64
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS unknown 02/02/2022
RIP: 0010:__text_poke+0x2aa/0x450
Call Trace:
 <TASK>
 smp_text_poke_batch_finish+0x2a7/0x320
 __static_call_transform+0xb7/0x220
 arch_static_call_transform+0x5b/0xb0
 __static_call_init+0xe9/0x270
 static_call_module_notify+0x11f/0x150
Quoted 8 lines
 notifier_call_chain+0x61/0xe0
 blocking_notifier_call_chain_robust+0x63/0xc0
 load_module+0x1c92/0x20c0
 init_module_from_file+0xd8/0x140
 idempotent_init_module+0x100/0x2f0
 __x64_sys_finit_module+0x71/0xe0
 do_syscall_64+0xe1/0x610
 entry_SYSCALL_64_after_hwframe+0x76/0x7e

which matches the following BUG_ON() in alternative.c:

	/*
	 * If something went wrong, crash and burn since recovery paths are not
	 * implemented.
	 */
	BUG_ON(!pages[0] || (cross_page_boundary && !pages[1]));

This can happen if vmalloc_to_page() fails, for any reason. Such can happen if text poking races with CPA, which can possibly result in the collapsing of page tables (or breaking of PMD hugepages). It is not a problem for most users of vmalloc_to_page() (they solely own the vmalloc'd range) but, when CONFIG_ARCH_HAS_EXECMEM_ROX=y, various modules own a single execmem vmalloc range, and can call set_memory_*() in parallel on it. This can happen to race against __text_poke and cause havoc in vmalloc_to_page().

Fix it by excluding against CPA using the init_mm mmap read lock.

[ dhansen: Fix up SoB ordering. The actual code flow here was:
Pedro=>Lorenzo=>Mike=>Me which is reflected in the SoB chain now. I *believe* Mike simply picked up Lorenzo's update to Pedro's post from the Link ]

---
 arch/x86/kernel/alternative.c | 39 +++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)
arch/x86/kernel/alternative.c +36 -3
index 91b1cdd..add62db 100644
@@ -6,6 +6,9 @@
 #include <linux/vmalloc.h>
 #include <linux/memory.h>
 #include <linux/execmem.h>
+#include <linux/cleanup.h>
+#include <linux/kgdb.h>
+#include <linux/mmap_lock.h>
 
 #include <asm/text-patching.h>
 #include <asm/insn.h>
@@ -2372,6 +2375,38 @@ static void text_poke_memset(void *dst, const void *src, size_t len)
 
 typedef void text_poke_f(void *dst, const void *src, size_t len);
 
+static void __poke_vmalloc_pages(struct page **pages, void *addr,
+				 bool cross_page_boundary)
+{
+	pages[0] = vmalloc_to_page(addr);
+	if (cross_page_boundary)
+		pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+}
+
+static void poke_vmalloc_pages(struct page **pages, void *addr,
+			       bool cross_page_boundary)
+{
+	if (in_dbg_master()) {
+		/*
+		 * If called from kgdb cannot sleep, but all other CPUs stopped
+		 * anyway so safe to proceed without locks
+		 */
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	} else {
+		/*
+		 * execmem ROX ranges are shared between modules and can be
+		 * collapsed to huge PMD entries, and this collapse can happen
+		 * concurrently with a racing set_memory_rox().
+		 *
+		 * Prevent vmalloc_to_page() from racing by acquiring an
+		 * init_mm read lock which pairs with the init_mm write lock in
+		 * cpa_collapse_large_pages().
+		 */
+		guard(mmap_read_lock)(&init_mm);
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	}
+}
+
 static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t len)
 {
 	bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
@@ -2389,9 +2424,7 @@ static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l
 	BUG_ON(!after_bootmem);
 
 	if (!core_kernel_text((unsigned long)addr)) {
-		pages[0] = vmalloc_to_page(addr);
-		if (cross_page_boundary)
-			pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+		poke_vmalloc_pages(pages, addr, cross_page_boundary);
 	} else {
 		pages[0] = virt_to_page(addr);
 		WARN_ON(!PageReserved(pages[0]));
merge bot Applied
#39 lore raw

[tip: x86/urgent] x86/alternatives: Exclude text poking against change_page_attr()

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     1587d3394e254639cc36516256031334095e6ef3
Gitweb:        https://git.kernel.org/tip/1587d3394e254639cc36516256031334095e6ef3
Author:        Pedro Falcato <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:26 +03:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 09 Sep 2026 08:39:10 +02:00

x86/alternatives: Exclude text poking against change_page_attr()

Quoted 1 line

From time to time, the following BUG can be observed

in the x86 alternatives patching code [0]:

Quoted 12 lines

kernel BUG at arch/x86/kernel/alternative.c:2576!

Oops: invalid opcode: 0000 [#1] SMP NOPTI
CPU: 0 UID: 0 PID: 355 Comm: (udev-worker) Not tainted 7.1.3-1-default #1 PREEMPT(full) openSUSE Tumbleweed  8c1795b03ec64f997e57a8ad38b1161e3b98da64
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS unknown 02/02/2022
RIP: 0010:__text_poke+0x2aa/0x450
Call Trace:
 <TASK>
 smp_text_poke_batch_finish+0x2a7/0x320
 __static_call_transform+0xb7/0x220
 arch_static_call_transform+0x5b/0xb0
 __static_call_init+0xe9/0x270
 static_call_module_notify+0x11f/0x150
Quoted 8 lines
 notifier_call_chain+0x61/0xe0
 blocking_notifier_call_chain_robust+0x63/0xc0
 load_module+0x1c92/0x20c0
 init_module_from_file+0xd8/0x140
 idempotent_init_module+0x100/0x2f0
 __x64_sys_finit_module+0x71/0xe0
 do_syscall_64+0xe1/0x610
 entry_SYSCALL_64_after_hwframe+0x76/0x7e

which matches the following BUG_ON() in alternative.c:

	/*
	 * If something went wrong, crash and burn since recovery paths are not
	 * implemented.
	 */
	BUG_ON(!pages[0] || (cross_page_boundary && !pages[1]));

This can happen if vmalloc_to_page() fails, for any reason. Such can happen if text poking races with CPA, which can possibly result in the collapsing of page tables (or breaking of PMD hugepages). It is not a problem for most users of vmalloc_to_page() (they solely own the vmalloc'd range) but, when CONFIG_ARCH_HAS_EXECMEM_ROX=y, various modules own a single execmem vmalloc range, and can call set_memory_*() in parallel on it. This can happen to race against __text_poke and cause havoc in vmalloc_to_page().

Fix it by excluding against CPA using the init_mm mmap read lock.

[ dhansen: Fix up SoB ordering. The actual code flow here was:
Pedro=>Lorenzo=>Mike=>Me which is reflected in the SoB chain now. I *believe* Mike simply picked up Lorenzo's update to Pedro's post from the Link ]

---
 arch/x86/kernel/alternative.c | 39 +++++++++++++++++++++++++++++++---
 1 file changed, 36 insertions(+), 3 deletions(-)
arch/x86/kernel/alternative.c +36 -3
index 91b1cdd..add62db 100644
@@ -6,6 +6,9 @@
 #include <linux/vmalloc.h>
 #include <linux/memory.h>
 #include <linux/execmem.h>
+#include <linux/cleanup.h>
+#include <linux/kgdb.h>
+#include <linux/mmap_lock.h>
 
 #include <asm/text-patching.h>
 #include <asm/insn.h>
@@ -2372,6 +2375,38 @@ static void text_poke_memset(void *dst, const void *src, size_t len)
 
 typedef void text_poke_f(void *dst, const void *src, size_t len);
 
+static void __poke_vmalloc_pages(struct page **pages, void *addr,
+				 bool cross_page_boundary)
+{
+	pages[0] = vmalloc_to_page(addr);
+	if (cross_page_boundary)
+		pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+}
+
+static void poke_vmalloc_pages(struct page **pages, void *addr,
+			       bool cross_page_boundary)
+{
+	if (in_dbg_master()) {
+		/*
+		 * If called from kgdb cannot sleep, but all other CPUs stopped
+		 * anyway so safe to proceed without locks
+		 */
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	} else {
+		/*
+		 * execmem ROX ranges are shared between modules and can be
+		 * collapsed to huge PMD entries, and this collapse can happen
+		 * concurrently with a racing set_memory_rox().
+		 *
+		 * Prevent vmalloc_to_page() from racing by acquiring an
+		 * init_mm read lock which pairs with the init_mm write lock in
+		 * cpa_collapse_large_pages().
+		 */
+		guard(mmap_read_lock)(&init_mm);
+		__poke_vmalloc_pages(pages, addr, cross_page_boundary);
+	}
+}
+
 static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t len)
 {
 	bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE;
@@ -2389,9 +2424,7 @@ static void *__text_poke(text_poke_f func, void *addr, const void *src, size_t l
 	BUG_ON(!after_bootmem);
 
 	if (!core_kernel_text((unsigned long)addr)) {
-		pages[0] = vmalloc_to_page(addr);
-		if (cross_page_boundary)
-			pages[1] = vmalloc_to_page(addr + PAGE_SIZE);
+		poke_vmalloc_pages(pages, addr, cross_page_boundary);
 	} else {
 		pages[0] = virt_to_page(addr);
 		WARN_ON(!PageReserved(pages[0]));
#40 lore raw

PATCH v2 4/5 x86/mm/pat: allocate split page tables as kernel page tables

In reply to Mike Rapoport

From: "Lorenzo Stoakes (ARM)" <[email protected]>

When splitting a large page in CPA in __split_large_page() we allocate a PTE directly without going through the standard page table allocation routines such as pte_alloc_one_kernel().

This means the page table constructor is never called nor is the page table marked as a kernel page table.

The former results in the folio associated with the page table not being marked as a page table (__pagetable_ctor() is never called thus neither is __folio_set_pgtable()) nor are statistics updated to reflect it (lruvec_stat_add_folio() is never called).

The latter issue of failing to mark the page table as a kernel page table (ptdesc_set_kernel() is never called) is far more problematic.

Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") kernel page table freeing has been batched and since the subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries for kernel address space") IOTLB cache entries for kernel page tables have been invalidated upon being freed.

Since split page tables are freed without this invalidation, the IOTLB can contain stale entries for them.

Resolve the issue by using the ordinary PTE allocation API at split time.

This results in these kernel page tables invoking a page table constructor, and thus requires a page table destructor.

Since we cannot assume one is always present (early allocated direct map page tables are not marked as such), we conditionally call pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set, otherwise we free the page table via pagetable_free().

Regardless of which path is taken page tables marked as kernel page tables, which now includes split page tables, take the correct route through pagetable_free_kernel().

There is a user-visible side effect in that split page tables will appear in nr_page_table_pages in /proc/vmstat (as do other kernel page tables allocated after early boot), however this is a positive change.

This issue started being markedly problematic after commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so choose this as the Fixes target.

Fixes 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")
---
 arch/x86/mm/pat/set_memory.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)
arch/x86/mm/pat/set_memory.c +16 -9
index 2d04a4bf34aa..fbc418dfc597 100644
@@ -441,7 +441,15 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 
 	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
 		list_del(&ptdesc->pt_list);
-		pagetable_free(ptdesc);
+		/*
+		 * Only early alloc'd direct map should not be flagged PG_table
+		 * here and those shouldn't be collapsed. However be abundantly
+		 * cautious and handle the !PG_table case too.
+		 */
+		if (PageTable((ptdesc_page(ptdesc))))
+			pagetable_dtor_free(ptdesc);
+		else
+			pagetable_free(ptdesc);
 	}
 }
 
@@ -1134,11 +1142,10 @@ static void split_set_pte(struct cpa_data *cpa, pte_t *pte, unsigned long pfn,
 
 static int
 __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
-		   struct ptdesc *ptdesc)
+		   pte_t *pbase)
 {
 	unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc = 1;
-	struct page *base = ptdesc_page(ptdesc);
-	pte_t *pbase = (pte_t *)page_address(base);
+	struct page *base = virt_to_page(pbase);
 	unsigned int i, level;
 	pgprot_t ref_prot;
 	bool nx, rw;
@@ -1238,20 +1245,20 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
 static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 			    unsigned long address)
 {
-	struct ptdesc *ptdesc;
+	pte_t *pte;
 
 	spin_unlock(&cpa_lock);
 	if (cpa->init_mm_read_locked)
 		mmap_read_unlock(&init_mm);
-	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	pte = pte_alloc_one_kernel(&init_mm);
 	if (cpa->init_mm_read_locked)
 		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
-	if (!ptdesc)
+	if (!pte)
 		return -ENOMEM;
 
-	if (__split_large_page(cpa, kpte, address, ptdesc))
-		pagetable_free(ptdesc);
+	if (__split_large_page(cpa, kpte, address, pte))
+		pte_free_kernel(&init_mm, pte);
 
 	return 0;
 }
Signature · 1 line
2.53.0
#41 lore raw

[tip: x86/urgent] x86/mm/pat: Allocate split page tables as kernel page tables

The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     83a2490f0f1e173bb10f0d076a546cf1cd36f609
Gitweb:        https://git.kernel.org/tip/83a2490f0f1e173bb10f0d076a546cf1cd36f609
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:27 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:17:53 -07:00

x86/mm/pat: Allocate split page tables as kernel page tables

PTEs are allocated directly without going through the standard page table allocation routines such as pte_alloc_one_kernel() when splitting a large page in CPA in __split_large_page().

This means the page table constructor is never called nor is the page table marked as a kernel page table.

The former results in the folio associated with the page table not being marked as a page table (__pagetable_ctor() is never called thus neither is __folio_set_pgtable()) nor are statistics updated to reflect it (lruvec_stat_add_folio() is never called).

The latter issue of failing to mark the page table as a kernel page table (ptdesc_set_kernel() is never called) is far more problematic.

Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") kernel page table freeing has been batched and since the subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries for kernel address space") IOTLB cache entries for kernel page tables have been invalidated upon being freed.

Since split page tables are freed without this invalidation, the IOTLB can contain stale entries for them.

Resolve the issue by using the ordinary PTE allocation API at split time.

This results in these kernel page tables invoking a page table constructor, and thus requires a page table destructor.

Since page table destructors are not always present (early allocated direct map page tables are not marked as such), conditionally call pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set. Otherwise, free the page table via pagetable_free().

Regardless of which path is taken page tables marked as kernel page tables, which now includes split page tables, take the correct route through pagetable_free_kernel().

There is a user-visible side effect in that split page tables will appear in nr_page_table_pages in /proc/vmstat (as do other kernel page tables allocated after early boot), however this is a positive change.

This issue started being markedly problematic after commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so choose this as the Fixes target.

[ dhansen: changelog tweaks for tip style ]

---
 arch/x86/mm/pat/set_memory.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)
arch/x86/mm/pat/set_memory.c +16 -9
index cb5d6d6..4652487 100644
@@ -441,7 +441,15 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 
 	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
 		list_del(&ptdesc->pt_list);
-		pagetable_free(ptdesc);
+		/*
+		 * Only early alloc'd direct map should not be flagged PG_table
+		 * here and those shouldn't be collapsed. However be abundantly
+		 * cautious and handle the !PG_table case too.
+		 */
+		if (PageTable((ptdesc_page(ptdesc))))
+			pagetable_dtor_free(ptdesc);
+		else
+			pagetable_free(ptdesc);
 	}
 }
 
@@ -1134,11 +1142,10 @@ set:
 
 static int
 __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
-		   struct ptdesc *ptdesc)
+		   pte_t *pbase)
 {
 	unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc = 1;
-	struct page *base = ptdesc_page(ptdesc);
-	pte_t *pbase = (pte_t *)page_address(base);
+	struct page *base = virt_to_page(pbase);
 	unsigned int i, level;
 	pgprot_t ref_prot;
 	bool nx, rw;
@@ -1238,20 +1245,20 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
 static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 			    unsigned long address)
 {
-	struct ptdesc *ptdesc;
+	pte_t *pte;
 
 	spin_unlock(&cpa_lock);
 	if (cpa->init_mm_read_locked)
 		mmap_read_unlock(&init_mm);
-	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	pte = pte_alloc_one_kernel(&init_mm);
 	if (cpa->init_mm_read_locked)
 		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
-	if (!ptdesc)
+	if (!pte)
 		return -ENOMEM;
 
-	if (__split_large_page(cpa, kpte, address, ptdesc))
-		pagetable_free(ptdesc);
+	if (__split_large_page(cpa, kpte, address, pte))
+		pte_free_kernel(&init_mm, pte);
 
 	return 0;
 }
#42 lore raw

[tip: x86/urgent] x86/mm/pat: Allocate split page tables as kernel page tables

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     0e33126d5def407397deaf617559a1a2a7f4b1ae
Gitweb:        https://git.kernel.org/tip/0e33126d5def407397deaf617559a1a2a7f4b1ae
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:27 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Tue, 01 Sep 2026 14:58:43 -07:00

x86/mm/pat: Allocate split page tables as kernel page tables

A PTE is allocated directly without going through the standard page table allocation routines (such as pte_alloc_one_kernel()) when the CPA code splits a large page (__split_large_page()).

This means the page table constructor is never called nor is the page table marked as a kernel page table.

The former results in the folio associated with the page table not being marked as a page table (__pagetable_ctor() is never called thus neither is __folio_set_pgtable()) nor are statistics updated to reflect it (lruvec_stat_add_folio() is never called).

The latter issue of failing to mark the page table as a kernel page table (ptdesc_set_kernel() is never called) is far more problematic.

Since commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") kernel page table freeing has been batched and since the subsequent commit e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries for kernel address space") IOTLB cache entries for kernel page tables have been invalidated upon being freed.

Since split page tables are freed without this invalidation, the IOTLB can contain stale entries for them.

Resolve the issue by using the ordinary PTE allocation API at split time.

This results in these kernel page tables invoking a page table constructor, and thus requires a page table destructor.

Destructors are not always present, like for early allocated direct map page tables). Conditionally call pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set, otherwise we free the page table via pagetable_free().

Regardless of which path is taken page tables marked as kernel page tables, which now includes split page tables, take the correct route through pagetable_free_kernel().

There is a user-visible side effect in that split page tables will appear in nr_page_table_pages in /proc/vmstat (as do other kernel page tables allocated after early boot), however this is a positive change.

This issue started being markedly problematic after commit 5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables") so choose this as the Fixes target.

[ dhansen: rephrase in imperative mood ]

---
 arch/x86/mm/pat/set_memory.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)
arch/x86/mm/pat/set_memory.c +16 -9
index cb5d6d6..4652487 100644
@@ -441,7 +441,15 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 
 	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
 		list_del(&ptdesc->pt_list);
-		pagetable_free(ptdesc);
+		/*
+		 * Only early alloc'd direct map should not be flagged PG_table
+		 * here and those shouldn't be collapsed. However be abundantly
+		 * cautious and handle the !PG_table case too.
+		 */
+		if (PageTable((ptdesc_page(ptdesc))))
+			pagetable_dtor_free(ptdesc);
+		else
+			pagetable_free(ptdesc);
 	}
 }
 
@@ -1134,11 +1142,10 @@ set:
 
 static int
 __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
-		   struct ptdesc *ptdesc)
+		   pte_t *pbase)
 {
 	unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc = 1;
-	struct page *base = ptdesc_page(ptdesc);
-	pte_t *pbase = (pte_t *)page_address(base);
+	struct page *base = virt_to_page(pbase);
 	unsigned int i, level;
 	pgprot_t ref_prot;
 	bool nx, rw;
@@ -1238,20 +1245,20 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
 static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 			    unsigned long address)
 {
-	struct ptdesc *ptdesc;
+	pte_t *pte;
 
 	spin_unlock(&cpa_lock);
 	if (cpa->init_mm_read_locked)
 		mmap_read_unlock(&init_mm);
-	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	pte = pte_alloc_one_kernel(&init_mm);
 	if (cpa->init_mm_read_locked)
 		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
-	if (!ptdesc)
+	if (!pte)
 		return -ENOMEM;
 
-	if (__split_large_page(cpa, kpte, address, ptdesc))
-		pagetable_free(ptdesc);
+	if (__split_large_page(cpa, kpte, address, pte))
+		pte_free_kernel(&init_mm, pte);
 
 	return 0;
 }
#43 lore raw

[tip: x86/mm] x86/mm/pat: Allocate split page tables as kernel page tables

In reply to Mike Rapoport
The following commit has been merged into the x86/mm branch of tip:

The following commit has been merged into the x86/mm branch of tip:

Commit-ID:     394562a5546aaaeabef6baef5715adf59b71a7c4
Gitweb:        https://git.kernel.org/tip/394562a5546aaaeabef6baef5715adf59b71a7c4
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:27 +03:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Tue, 08 Sep 2026 09:12:55 +02:00

x86/mm/pat: Allocate split page tables as kernel page tables

A PTE is allocated directly without going through the standard page table allocation routines (such as pte_alloc_one_kernel()) when the CPA code splits a large page (__split_large_page()).

This means the page table constructor is never called nor is the page table marked as a kernel page table.

The former results in the folio associated with the page table not being marked as a page table (__pagetable_ctor() is never called thus neither is __folio_set_pgtable()) nor are statistics updated to reflect it (lruvec_stat_add_folio() is never called).

The latter issue of failing to mark the page table as a kernel page table (ptdesc_set_kernel() is never called) is far more problematic.

Since commit:

5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")

kernel page table freeing has been batched and since the subsequent commit:

e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries for kernel address space")

IOTLB cache entries for kernel page tables have been invalidated upon being freed.

Since split page tables are freed without this invalidation, the IOTLB can contain stale entries for them.

Resolve the issue by using the ordinary PTE allocation API at split time.

This results in these kernel page tables invoking a page table constructor, and thus requires a page table destructor.

Destructors are not always present, like for early allocated direct map page tables). Conditionally call pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set, otherwise we free the page table via pagetable_free().

Regardless of which path is taken page tables marked as kernel page tables, which now includes split page tables, take the correct route through pagetable_free_kernel().

There is a user-visible side effect in that split page tables will appear in nr_page_table_pages in /proc/vmstat (as do other kernel page tables allocated after early boot), however this is a positive change.

This issue started being markedly problematic after commit:

5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")

so choose this as the Fixes target.

[ dhansen: rephrase in imperative mood ]

---
 arch/x86/mm/pat/set_memory.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)
arch/x86/mm/pat/set_memory.c +16 -9
index cb5d6d6..4652487 100644
@@ -441,7 +441,15 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 
 	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
 		list_del(&ptdesc->pt_list);
-		pagetable_free(ptdesc);
+		/*
+		 * Only early alloc'd direct map should not be flagged PG_table
+		 * here and those shouldn't be collapsed. However be abundantly
+		 * cautious and handle the !PG_table case too.
+		 */
+		if (PageTable((ptdesc_page(ptdesc))))
+			pagetable_dtor_free(ptdesc);
+		else
+			pagetable_free(ptdesc);
 	}
 }
 
@@ -1134,11 +1142,10 @@ set:
 
 static int
 __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
-		   struct ptdesc *ptdesc)
+		   pte_t *pbase)
 {
 	unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc = 1;
-	struct page *base = ptdesc_page(ptdesc);
-	pte_t *pbase = (pte_t *)page_address(base);
+	struct page *base = virt_to_page(pbase);
 	unsigned int i, level;
 	pgprot_t ref_prot;
 	bool nx, rw;
@@ -1238,20 +1245,20 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
 static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 			    unsigned long address)
 {
-	struct ptdesc *ptdesc;
+	pte_t *pte;
 
 	spin_unlock(&cpa_lock);
 	if (cpa->init_mm_read_locked)
 		mmap_read_unlock(&init_mm);
-	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	pte = pte_alloc_one_kernel(&init_mm);
 	if (cpa->init_mm_read_locked)
 		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
-	if (!ptdesc)
+	if (!pte)
 		return -ENOMEM;
 
-	if (__split_large_page(cpa, kpte, address, ptdesc))
-		pagetable_free(ptdesc);
+	if (__split_large_page(cpa, kpte, address, pte))
+		pte_free_kernel(&init_mm, pte);
 
 	return 0;
 }
#44 lore raw

[tip: x86/urgent] x86/mm/pat: Allocate split page tables as kernel page tables

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     fa138406b1cb4e72170a5dfc011cbc76bf4795f7
Gitweb:        https://git.kernel.org/tip/fa138406b1cb4e72170a5dfc011cbc76bf4795f7
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:27 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Tue, 08 Sep 2026 15:43:07 -07:00

x86/mm/pat: Allocate split page tables as kernel page tables

A PTE is allocated directly without going through the standard page table allocation routines (such as pte_alloc_one_kernel()) when the CPA code splits a large page (__split_large_page()).

This means the page table constructor is never called nor is the page table marked as a kernel page table.

The former results in the folio associated with the page table not being marked as a page table (__pagetable_ctor() is never called thus neither is __folio_set_pgtable()) nor are statistics updated to reflect it (lruvec_stat_add_folio() is never called).

The latter issue of failing to mark the page table as a kernel page table (ptdesc_set_kernel() is never called) is far more problematic.

Since commit:

5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")

kernel page table freeing has been batched and since the subsequent commit:

e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries for kernel address space")

IOTLB cache entries for kernel page tables have been invalidated upon being freed.

Since split page tables are freed without this invalidation, the IOTLB can contain stale entries for them.

Resolve the issue by using the ordinary PTE allocation API at split time.

This results in these kernel page tables invoking a page table constructor, and thus requires a page table destructor.

Destructors are not always present, like for early allocated direct map page tables). Conditionally call pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set, otherwise we free the page table via pagetable_free().

Regardless of which path is taken page tables marked as kernel page tables, which now includes split page tables, take the correct route through pagetable_free_kernel().

There is a user-visible side effect in that split page tables will appear in nr_page_table_pages in /proc/vmstat (as do other kernel page tables allocated after early boot), however this is a positive change.

This issue started being markedly problematic after commit:

5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")

so choose this as the Fixes target.

[ dhansen: rephrase in imperative mood ]

---
 arch/x86/mm/pat/set_memory.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)
arch/x86/mm/pat/set_memory.c +16 -9
index cb5d6d6..4652487 100644
@@ -441,7 +441,15 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 
 	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
 		list_del(&ptdesc->pt_list);
-		pagetable_free(ptdesc);
+		/*
+		 * Only early alloc'd direct map should not be flagged PG_table
+		 * here and those shouldn't be collapsed. However be abundantly
+		 * cautious and handle the !PG_table case too.
+		 */
+		if (PageTable((ptdesc_page(ptdesc))))
+			pagetable_dtor_free(ptdesc);
+		else
+			pagetable_free(ptdesc);
 	}
 }
 
@@ -1134,11 +1142,10 @@ set:
 
 static int
 __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
-		   struct ptdesc *ptdesc)
+		   pte_t *pbase)
 {
 	unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc = 1;
-	struct page *base = ptdesc_page(ptdesc);
-	pte_t *pbase = (pte_t *)page_address(base);
+	struct page *base = virt_to_page(pbase);
 	unsigned int i, level;
 	pgprot_t ref_prot;
 	bool nx, rw;
@@ -1238,20 +1245,20 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
 static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 			    unsigned long address)
 {
-	struct ptdesc *ptdesc;
+	pte_t *pte;
 
 	spin_unlock(&cpa_lock);
 	if (cpa->init_mm_read_locked)
 		mmap_read_unlock(&init_mm);
-	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	pte = pte_alloc_one_kernel(&init_mm);
 	if (cpa->init_mm_read_locked)
 		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
-	if (!ptdesc)
+	if (!pte)
 		return -ENOMEM;
 
-	if (__split_large_page(cpa, kpte, address, ptdesc))
-		pagetable_free(ptdesc);
+	if (__split_large_page(cpa, kpte, address, pte))
+		pte_free_kernel(&init_mm, pte);
 
 	return 0;
 }
#45 lore raw

[tip: x86/urgent] x86/mm/pat: Allocate split page tables as kernel page tables

In reply to Mike Rapoport
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     9e4a3ec3411bb6bb59e3c1f29b75609f1e87aac4
Gitweb:        https://git.kernel.org/tip/9e4a3ec3411bb6bb59e3c1f29b75609f1e87aac4
Author:        Lorenzo Stoakes (ARM) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:27 +03:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Wed, 09 Sep 2026 08:39:11 +02:00

x86/mm/pat: Allocate split page tables as kernel page tables

A PTE is allocated directly without going through the standard page table allocation routines (such as pte_alloc_one_kernel()) when the CPA code splits a large page (__split_large_page()).

This means the page table constructor is never called nor is the page table marked as a kernel page table.

The former results in the folio associated with the page table not being marked as a page table (__pagetable_ctor() is never called thus neither is __folio_set_pgtable()) nor are statistics updated to reflect it (lruvec_stat_add_folio() is never called).

The latter issue of failing to mark the page table as a kernel page table (ptdesc_set_kernel() is never called) is far more problematic.

Since commit:

5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")

kernel page table freeing has been batched and since the subsequent commit:

e37d5a2d60a3 ("iommu/sva: invalidate stale IOTLB entries for kernel address space")

IOTLB cache entries for kernel page tables have been invalidated upon being freed.

Since split page tables are freed without this invalidation, the IOTLB can contain stale entries for them.

Resolve the issue by using the ordinary PTE allocation API at split time.

This results in these kernel page tables invoking a page table constructor, and thus requires a page table destructor.

Destructors are not always present, like for early allocated direct map page tables). Conditionally call pagetable_dtor_free() if the PG_table folio flag for the ptdesc is set, otherwise we free the page table via pagetable_free().

Regardless of which path is taken page tables marked as kernel page tables, which now includes split page tables, take the correct route through pagetable_free_kernel().

There is a user-visible side effect in that split page tables will appear in nr_page_table_pages in /proc/vmstat (as do other kernel page tables allocated after early boot), however this is a positive change.

This issue started being markedly problematic after commit:

5ba2f0a15564 ("mm: introduce deferred freeing for kernel page tables")

so choose this as the Fixes target.

[ dhansen: rephrase in imperative mood ]

---
 arch/x86/mm/pat/set_memory.c | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)
arch/x86/mm/pat/set_memory.c +16 -9
index cb5d6d6..4652487 100644
@@ -441,7 +441,15 @@ static void __cpa_collapse_large_pages(struct cpa_data *cpa)
 
 	list_for_each_entry_safe(ptdesc, tmp, &pgtables, pt_list) {
 		list_del(&ptdesc->pt_list);
-		pagetable_free(ptdesc);
+		/*
+		 * Only early alloc'd direct map should not be flagged PG_table
+		 * here and those shouldn't be collapsed. However be abundantly
+		 * cautious and handle the !PG_table case too.
+		 */
+		if (PageTable((ptdesc_page(ptdesc))))
+			pagetable_dtor_free(ptdesc);
+		else
+			pagetable_free(ptdesc);
 	}
 }
 
@@ -1134,11 +1142,10 @@ set:
 
 static int
 __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
-		   struct ptdesc *ptdesc)
+		   pte_t *pbase)
 {
 	unsigned long lpaddr, lpinc, ref_pfn, pfn, pfninc = 1;
-	struct page *base = ptdesc_page(ptdesc);
-	pte_t *pbase = (pte_t *)page_address(base);
+	struct page *base = virt_to_page(pbase);
 	unsigned int i, level;
 	pgprot_t ref_prot;
 	bool nx, rw;
@@ -1238,20 +1245,20 @@ __split_large_page(struct cpa_data *cpa, pte_t *kpte, unsigned long address,
 static int split_large_page(struct cpa_data *cpa, pte_t *kpte,
 			    unsigned long address)
 {
-	struct ptdesc *ptdesc;
+	pte_t *pte;
 
 	spin_unlock(&cpa_lock);
 	if (cpa->init_mm_read_locked)
 		mmap_read_unlock(&init_mm);
-	ptdesc = pagetable_alloc(GFP_KERNEL, 0);
+	pte = pte_alloc_one_kernel(&init_mm);
 	if (cpa->init_mm_read_locked)
 		mmap_read_lock(&init_mm);
 	spin_lock(&cpa_lock);
-	if (!ptdesc)
+	if (!pte)
 		return -ENOMEM;
 
-	if (__split_large_page(cpa, kpte, address, ptdesc))
-		pagetable_free(ptdesc);
+	if (__split_large_page(cpa, kpte, address, pte))
+		pte_free_kernel(&init_mm, pte);
 
 	return 0;
 }
#46 lore raw

PATCH v2 5/5 x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()

In reply to Mike Rapoport

lookup_address_in_pgd_attr() accumulates the effective NX and RW bits of the walked page table levels so that verify_rwx() can detect mappings that are both writable and executable.

The RW bits are folded into a bool with

	rw &= pXd_flags(*pXd) & _PAGE_RW;

but _PAGE_RW is 0x2. So consider the accumulation line:

        rw &= pXd_flags(*pXd) & _PAGE_RW;

where rw=0x1 and the right side evaluates down to 0x2. It'll end up doing:

        rw = 0x1 & 0x2

and rw always ends up 0.

This way rw becomes false at the first level walked, regardless of the actual permissions, and verify_rwx() treats every mapping as non-writable and never reports a W^X violation.

Add double negation to the right side to normalize the _PAGE_RW flag to 0 or 1.

Fixes ceb647b4b529 ("x86/pat: Introduce lookup_address_in_pgd_attr()")
Assisted-by Copilot:claude-opus-4.8
---
 arch/x86/mm/pat/set_memory.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/mm/pat/set_memory.c +4 -4
index fbc418dfc597..430d0b448371 100644
@@ -754,7 +754,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_512G;
 	*nx |= pgd_flags(*pgd) & _PAGE_NX;
-	*rw &= pgd_flags(*pgd) & _PAGE_RW;
+	*rw &= !!(pgd_flags(*pgd) & _PAGE_RW);
 
 	p4d = p4d_offset(pgd, address);
 	if (p4d_none(*p4d))
@@ -765,7 +765,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_1G;
 	*nx |= p4d_flags(*p4d) & _PAGE_NX;
-	*rw &= p4d_flags(*p4d) & _PAGE_RW;
+	*rw &= !!(p4d_flags(*p4d) & _PAGE_RW);
 
 	pud = pud_offset(p4d, address);
 	if (pud_none(*pud))
@@ -776,7 +776,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_2M;
 	*nx |= pud_flags(*pud) & _PAGE_NX;
-	*rw &= pud_flags(*pud) & _PAGE_RW;
+	*rw &= !!(pud_flags(*pud) & _PAGE_RW);
 
 	pmd = pmd_offset(pud, address);
 	if (pmd_none(*pmd))
@@ -787,7 +787,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_4K;
 	*nx |= pmd_flags(*pmd) & _PAGE_NX;
-	*rw &= pmd_flags(*pmd) & _PAGE_RW;
+	*rw &= !!(pmd_flags(*pmd) & _PAGE_RW);
 
 	return pte_offset_kernel(pmd, address);
 }
Signature · 1 line
2.53.0
maintainer Reviewed-by on v2 5/5
#47 lore raw

Re: [PATCH v2 5/5] x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()

Quoted 25 lines · Mike Rapoport (Microsoft)

lookup_address_in_pgd_attr() accumulates the effective NX and RW bits of the walked page table levels so that verify_rwx() can detect mappings that are both writable and executable.

The RW bits are folded into a bool with

	rw &= pXd_flags(*pXd) & _PAGE_RW;

but _PAGE_RW is 0x2. So consider the accumulation line:

        rw &= pXd_flags(*pXd) & _PAGE_RW;

where rw=0x1 and the right side evaluates down to 0x2. It'll end up doing:

        rw = 0x1 & 0x2

and rw always ends up 0.

This way rw becomes false at the first level walked, regardless of the actual permissions, and verify_rwx() treats every mapping as non-writable and never reports a W^X violation.

Add double negation to the right side to normalize the _PAGE_RW flag to 0 or 1.

Quoted 6 lines
Fixes ceb647b4b529 ("x86/pat: Introduce lookup_address_in_pgd_attr()")
Assisted-by Copilot:claude-opus-4.8

Yikes, good find! LGTM so:

Quoted 46 lines
---
 arch/x86/mm/pat/set_memory.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/mm/pat/set_memory.c +4 -5
index fbc418dfc597..430d0b448371 100644
@@ -754,7 +754,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_512G;
 	*nx |= pgd_flags(*pgd) & _PAGE_NX;
-	*rw &= pgd_flags(*pgd) & _PAGE_RW;
+	*rw &= !!(pgd_flags(*pgd) & _PAGE_RW);
 
 	p4d = p4d_offset(pgd, address);
 	if (p4d_none(*p4d))
@@ -765,7 +765,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_1G;
 	*nx |= p4d_flags(*p4d) & _PAGE_NX;
-	*rw &= p4d_flags(*p4d) & _PAGE_RW;
+	*rw &= !!(p4d_flags(*p4d) & _PAGE_RW);
 
 	pud = pud_offset(p4d, address);
 	if (pud_none(*pud))
@@ -776,7 +776,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_2M;
 	*nx |= pud_flags(*pud) & _PAGE_NX;
-	*rw &= pud_flags(*pud) & _PAGE_RW;
+	*rw &= !!(pud_flags(*pud) & _PAGE_RW);
 
 	pmd = pmd_offset(pud, address);
 	if (pmd_none(*pmd))
@@ -787,7 +787,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_4K;
 	*nx |= pmd_flags(*pmd) & _PAGE_NX;
-	*rw &= pmd_flags(*pmd) & _PAGE_RW;
+	*rw &= !!(pmd_flags(*pmd) & _PAGE_RW);
 
 	return pte_offset_kernel(pmd, address);
 }
 
--
Quoted 2 lines

2.53.0

--
Cheers, Lorenzo

#48 lore raw

[tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

In reply to Mike Rapoport (Microsoft)
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     34cd1c931a365a6b81e00e743f9f260050104ff6
Gitweb:        https://git.kernel.org/tip/34cd1c931a365a6b81e00e743f9f260050104ff6
Author:        Mike Rapoport (Microsoft) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:28 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Mon, 31 Aug 2026 15:20:52 -07:00

x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

lookup_address_in_pgd_attr() accumulates the effective NX and RW bits of the walked page table levels so that verify_rwx() can detect mappings that are both writable and executable.

The RW bits are folded into a bool with

	rw &= pXd_flags(*pXd) & _PAGE_RW;

but _PAGE_RW is 0x2. So consider the accumulation line:

        rw &= pXd_flags(*pXd) & _PAGE_RW;

where rw=0x1 and the right side evaluates down to 0x2. It'll end up doing:

        rw = 0x1 & 0x2

and rw always ends up 0.

This way rw becomes false at the first level walked, regardless of the actual permissions, and verify_rwx() treats every mapping as non-writable and never reports a W^X violation.

Add double negation to the right side to normalize the _PAGE_RW flag to 0 or 1.

---
 arch/x86/mm/pat/set_memory.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/mm/pat/set_memory.c +4 -4
index 4652487..2266609 100644
@@ -754,7 +754,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_512G;
 	*nx |= pgd_flags(*pgd) & _PAGE_NX;
-	*rw &= pgd_flags(*pgd) & _PAGE_RW;
+	*rw &= !!(pgd_flags(*pgd) & _PAGE_RW);
 
 	p4d = p4d_offset(pgd, address);
 	if (p4d_none(*p4d))
@@ -765,7 +765,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_1G;
 	*nx |= p4d_flags(*p4d) & _PAGE_NX;
-	*rw &= p4d_flags(*p4d) & _PAGE_RW;
+	*rw &= !!(p4d_flags(*p4d) & _PAGE_RW);
 
 	pud = pud_offset(p4d, address);
 	if (pud_none(*pud))
@@ -776,7 +776,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_2M;
 	*nx |= pud_flags(*pud) & _PAGE_NX;
-	*rw &= pud_flags(*pud) & _PAGE_RW;
+	*rw &= !!(pud_flags(*pud) & _PAGE_RW);
 
 	pmd = pmd_offset(pud, address);
 	if (pmd_none(*pmd))
@@ -787,7 +787,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_4K;
 	*nx |= pmd_flags(*pmd) & _PAGE_NX;
-	*rw &= pmd_flags(*pmd) & _PAGE_RW;
+	*rw &= !!(pmd_flags(*pmd) & _PAGE_RW);
 
 	return pte_offset_kernel(pmd, address);
 }
#49 lore raw

[tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

In reply to Mike Rapoport (Microsoft)
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     453e7859443446b837d905d6f2983a76c867247d
Gitweb:        https://git.kernel.org/tip/453e7859443446b837d905d6f2983a76c867247d
Author:        Mike Rapoport (Microsoft) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:28 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Tue, 01 Sep 2026 15:03:18 -07:00

x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

lookup_address_in_pgd_attr() accumulates the effective NX and RW bits of the walked page table levels so that verify_rwx() can detect mappings that are both writable and executable.

The RW bits are folded into a bool with

	rw &= pXd_flags(*pXd) & _PAGE_RW;

but _PAGE_RW is 0x2. So consider the accumulation line:

        rw &= pXd_flags(*pXd) & _PAGE_RW;

where rw=0x1 and the right side evaluates down to 0x2. It'll end up doing:

        rw = 0x1 & 0x2

and rw always ends up 0.

This way rw becomes false at the first level walked, regardless of the actual permissions, and verify_rwx() treats every mapping as non-writable and never reports a W^X violation.

Add double negation to the right side to normalize the _PAGE_RW flag to 0 or 1.

---
 arch/x86/mm/pat/set_memory.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/mm/pat/set_memory.c +4 -4
index 4652487..2266609 100644
@@ -754,7 +754,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_512G;
 	*nx |= pgd_flags(*pgd) & _PAGE_NX;
-	*rw &= pgd_flags(*pgd) & _PAGE_RW;
+	*rw &= !!(pgd_flags(*pgd) & _PAGE_RW);
 
 	p4d = p4d_offset(pgd, address);
 	if (p4d_none(*p4d))
@@ -765,7 +765,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_1G;
 	*nx |= p4d_flags(*p4d) & _PAGE_NX;
-	*rw &= p4d_flags(*p4d) & _PAGE_RW;
+	*rw &= !!(p4d_flags(*p4d) & _PAGE_RW);
 
 	pud = pud_offset(p4d, address);
 	if (pud_none(*pud))
@@ -776,7 +776,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_2M;
 	*nx |= pud_flags(*pud) & _PAGE_NX;
-	*rw &= pud_flags(*pud) & _PAGE_RW;
+	*rw &= !!(pud_flags(*pud) & _PAGE_RW);
 
 	pmd = pmd_offset(pud, address);
 	if (pmd_none(*pmd))
@@ -787,7 +787,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_4K;
 	*nx |= pmd_flags(*pmd) & _PAGE_NX;
-	*rw &= pmd_flags(*pmd) & _PAGE_RW;
+	*rw &= !!(pmd_flags(*pmd) & _PAGE_RW);
 
 	return pte_offset_kernel(pmd, address);
 }
#50 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

Hi folks,

Quoted 83 lines · On Wed, Sep 02, 2026 at 06:33:31PM -0000, tip-bot2 for Mike Rapoport…

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     453e7859443446b837d905d6f2983a76c867247d
Gitweb:        https://git.kernel.org/tip/453e7859443446b837d905d6f2983a76c867247d
Author:        Mike Rapoport (Microsoft) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:28 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Tue, 01 Sep 2026 15:03:18 -07:00

x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

lookup_address_in_pgd_attr() accumulates the effective NX and RW bits of the walked page table levels so that verify_rwx() can detect mappings that are both writable and executable.

The RW bits are folded into a bool with

	rw &= pXd_flags(*pXd) & _PAGE_RW;

but _PAGE_RW is 0x2. So consider the accumulation line:

        rw &= pXd_flags(*pXd) & _PAGE_RW;

where rw=0x1 and the right side evaluates down to 0x2. It'll end up doing:

        rw = 0x1 & 0x2

and rw always ends up 0.

This way rw becomes false at the first level walked, regardless of the actual permissions, and verify_rwx() treats every mapping as non-writable and never reports a W^X violation.

Add double negation to the right side to normalize the _PAGE_RW flag to 0 or 1.

---
 arch/x86/mm/pat/set_memory.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/mm/pat/set_memory.c +3 -3
index 4652487..2266609 100644
@@ -754,7 +754,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_512G;
 	*nx |= pgd_flags(*pgd) & _PAGE_NX;
-	*rw &= pgd_flags(*pgd) & _PAGE_RW;
+	*rw &= !!(pgd_flags(*pgd) & _PAGE_RW);
 
 	p4d = p4d_offset(pgd, address);
 	if (p4d_none(*p4d))
@@ -765,7 +765,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_1G;
 	*nx |= p4d_flags(*p4d) & _PAGE_NX;
-	*rw &= p4d_flags(*p4d) & _PAGE_RW;
+	*rw &= !!(p4d_flags(*p4d) & _PAGE_RW);
 
 	pud = pud_offset(p4d, address);
 	if (pud_none(*pud))
@@ -776,7 +776,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_2M;
 	*nx |= pud_flags(*pud) & _PAGE_NX;
-	*rw &= pud_flags(*pud) & _PAGE_RW;
+	*rw &= !!(pud_flags(*pud) & _PAGE_RW);
 
 	pmd = pmd_offset(pud, address);
 	if (pmd_none(*pmd))

arch/x86/mm/pat/set_memory.c pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address, · line 793

Quoted 8 lines
arch/x86/mm/pat/set_memory.c +1 -1
@@ -787,7 +787,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_4K;
 	*nx |= pmd_flags(*pmd) & _PAGE_NX;
-	*rw &= pmd_flags(*pmd) & _PAGE_RW;
+	*rw &= !!(pmd_flags(*pmd) & _PAGE_RW);
 
 	return pte_offset_kernel(pmd, address);
 }

I just bisected the following warning on a couple of my test machines to commit 453e78594434 ("x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()") in next-20260904.

[Sep 4 21:36] CPA detected W^X violation: 8000000000000123 -> 0000000000000123 range: 0xffffffffc0400000 - 0xffffffffc0400fff PFN 100e00

  [  +0.000001] WARNING: arch/x86/mm/pat/set_memory.c:722 at __change_page_attr_set_clr+0xde7/0x1290, CPU#0: swapper/0/0
  [  +0.000005] Modules linked in:
  [  +0.000002] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 7.3.0-rc1-debug-00006-g453e78594434 #1 PREEMPT(full)  2950d432dd3910251071a66f3134fe0875432786
  [  +0.000001] Hardware name: ASUS System Product Name/PRIME Z590M-PLUS, BIOS 1801 12/26/2022
  [  +0.000001] RIP: 0010:__change_page_attr_set_clr+0xdff/0x1290
  [  +0.000002] Code: 80 7c 24 42 00 0f 85 3a 04 00 00 48 8d 3d 19 8d 79 02 49 89 d9 4c 89 e1 4c 89 d2 4c 89 f6 4d 8d 84 24 ff 0f 00 00 4c 89 14 24 <67> 48 0f b9 3a 4c 8b 14 24 48 8b 0d 81 44 bf 01 41 f6 c2 01 0f 85
  [  +0.000001] RSP: 0000:ffffffff87003c60 EFLAGS: 00010246
  [  +0.000002] RAX: 0000000000000002 RBX: 0000000000100e00 RCX: ffffffffc0400000
  [  +0.000000] RDX: 0000000000000123 RSI: 8000000000000123 RDI: ffffffff872e50c0
  [  +0.000001] RBP: 8000000100e00123 R08: ffffffffc0400fff R09: 0000000000100e00
  [  +0.000001] R10: 0000000000000123 R11: 0000000000000001 R12: ffffffffc0400000
  [  +0.000000] R13: 0000000100e00123 R14: 8000000000000123 R15: ffffffff87003d58
  [  +0.000001] FS:  0000000000000000(0000) GS:ffff8ad1777a7000(0000) knlGS:0000000000000000
  [  +0.000001] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
  [  +0.000001] CR2: ffff8ad0a4201000 CR3: 00000007e3022001 CR4: 0000000000770ef0
  [  +0.000001] PKRU: 55555554
  [  +0.000001] Call Trace:
  [  +0.000001]  <TASK>
  [  +0.000002]  ? _vm_unmap_aliases+0x219/0x280
  [  +0.000003]  change_page_attr_set_clr+0x161/0x250
  [  +0.000002]  ? events_sysfs_show+0x5d/0x80
  [  +0.000002]  set_memory_x+0x39/0x50
  [  +0.000002]  apply_retpolines+0x656/0x6d0
  [  +0.000002]  ? events_sysfs_show+0x5d/0x80
  [  +0.000002]  ? events_sysfs_show+0x6c/0x80
  [  +0.000001]  ? events_sysfs_show+0x62/0x80
  [  +0.000001]  alternative_instructions+0x3c/0xd0
  [  +0.000003]  arch_cpu_finalize_init+0x130/0x190
  [  +0.000003]  start_kernel+0x97d/0xa10
  [  +0.000002]  x86_64_start_reservations+0x24/0x30
  [  +0.000002]  x86_64_start_kernel+0xda/0xe0
  [  +0.000002]  common_startup_64+0x13e/0x151
  [  +0.000003]  </TASK>
  [  +0.000001] ---[ end trace 0000000000000000 ]---

# bad: [af5f12805e5cefa4fe68d6127c7e1fb78cd5535c] Add linux-next specific files for 20260904
# good: [421066905cbceca1f78cba5f7d92b4980317ab2b] Merge tag 'probes-fixes-v7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
git bisect start 'af5f12805e5cefa4fe68d6127c7e1fb78cd5535c' '421066905cbceca1f78cba5f7d92b4980317ab2b'
# bad: [7e9b8247e9129fa3803020d3262e2384a0cfc29a] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/hid/hid.git
git bisect bad 7e9b8247e9129fa3803020d3262e2384a0cfc29a
# bad: [c4faef7e83f9c601c1264c0fad20af0e00b5a52b] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git
git bisect bad c4faef7e83f9c601c1264c0fad20af0e00b5a52b
# good: [14bd0d1671d1655d33419b60c9c476d0590636dd] Merge branch 'drm-fixes' of https://gitlab.freedesktop.org/drm/kernel.git
git bisect good 14bd0d1671d1655d33419b60c9c476d0590636dd
# good: [8d6fa663f11a859304187a70c569ec0617aa2461] Merge https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git mm-unstable into for-next git bisect good 8d6fa663f11a859304187a70c569ec0617aa2461
# bad: [276cefa1ab31a5d4fe5e4446f5e504499cc03320] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/mm/linux.git
git bisect bad 276cefa1ab31a5d4fe5e4446f5e504499cc03320
# good: [aad09a60e0012e85b03540a12e1fd1a61646712b] Merge branch 'rust-fixes' of https://github.com/Rust-for-Linux/linux.git
git bisect good aad09a60e0012e85b03540a12e1fd1a61646712b
# bad: [586e57cb2f695099a15e72029804a0592e6ad566] Merge branch into tip/master: 'x86/urgent'
git bisect bad 586e57cb2f695099a15e72029804a0592e6ad566
# good: [223f1750ff141a7e59b2724b645f348ffdab12e9] Merge branch into tip/master: 'sched/urgent'
git bisect good 223f1750ff141a7e59b2724b645f348ffdab12e9
# good: [e679ba0983757e9567aeda97cc35c99241d420ee] x86/alternative: Exclude text poking against change_page_attr()
git bisect good e679ba0983757e9567aeda97cc35c99241d420ee
# bad: [453e7859443446b837d905d6f2983a76c867247d] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()
git bisect bad 453e7859443446b837d905d6f2983a76c867247d
# good: [0e33126d5def407397deaf617559a1a2a7f4b1ae] x86/mm/pat: Allocate split page tables as kernel page tables
git bisect good 0e33126d5def407397deaf617559a1a2a7f4b1ae

  # first 'bad' commit: [453e7859443446b837d905d6f2983a76c867247d] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

If there is any information I can provide to help debug this, I am happy to provide it.

Signature · 2 lines
Cheers,
Nathan
#51 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

Quoted 3 lines · Nathan Chancellor

I just bisected the following warning on a couple of my test machines to commit 453e78594434 ("x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()") in next-20260904.

...

Quoted 2 lines

If there is any information I can provide to help debug this, I am happy to provide it.

Well, the thing you bisected to is the patch that fixes the warning. The actual issue comes from:

        if (pages == &its_pages)
                set_memory_x((unsigned long)page, 1);

which its_alloc() does on an execmem allocation. Any chance you could get the output of:

/sys/kernel/debug/page_tables/current_kernel

for the page that page that it's complaining about? I'm curious if this is temporary or permanent. its_pages_protect() should be fixing this up at the end of things, so there would be no permanent harm just a temporary violation of CONFIG_STRICT_MODULE_RWX.

Peter, shouldn't we be using the text poking mm for things like this?

#52 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

Quoted 9 lines · Dave Hansen
Quoted 3 lines · Nathan Chancellor

I just bisected the following warning on a couple of my test machines to commit 453e78594434 ("x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()") in next-20260904.

...

Quoted 2 lines

If there is any information I can provide to help debug this, I am happy to provide it.

Well, the thing you bisected to is the patch that fixes the warning. The

Hmmm, I don't see a warning prior to this patch though. Do you mean that this patch is just revealing the bad behavior that you mention below?

Quoted 5 lines

actual issue comes from:

        if (pages == &its_pages)
                set_memory_x((unsigned long)page, 1);
Quoted 7 lines

which its_alloc() does on an execmem allocation. Any chance you could get the output of:

/sys/kernel/debug/page_tables/current_kernel

for the page that page that it's complaining about? I'm curious if this is temporary or permanent.

Sure. For

CPA detected W^X violation: 8000000000000123 -> 0000000000000123 range: 0xffffffffc0200000 - 0xffffffffc0200fff PFN 100e00

I think it is this line if I understand correctly?

0xffffffffc0200000-0xffffffffc0400000 2M ro PSE GLB x pmd

Signature · 2 lines
Cheers,
Nathan
#53 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

Quoted 5 lines · Nathan Chancellor
Quoted 4 lines · Dave Hansen
Quoted 3 lines · Nathan Chancellor

I just bisected the following warning on a couple of my test machines to commit 453e78594434 ("x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()") in next-20260904.

Quoted 8 lines
Quoted 5 lines

...

Quoted 2 lines

If there is any information I can provide to help debug this, I am happy to provide it.

Well, the thing you bisected to is the patch that fixes the warning. The

Hmmm, I don't see a warning prior to this patch though. Do you mean that this patch is just revealing the bad behavior that you mention below?

Yeah, "fixes" was a bad word to use.

The W^X detection has been broken for a while. The patch you bisected to fixes the W^X detection. It doesn't actually cause the issue.

Quoted 13 lines
Quoted 5 lines

actual issue comes from:

        if (pages == &its_pages)
                set_memory_x((unsigned long)page, 1);
Quoted 7 lines

which its_alloc() does on an execmem allocation. Any chance you could get the output of:

/sys/kernel/debug/page_tables/current_kernel

for the page that page that it's complaining about? I'm curious if this is temporary or permanent.

Quoted 7 lines

Sure. For

   CPA detected W^X violation: 8000000000000123 -> 0000000000000123 range: 0xffffffffc0200000 - 0xffffffffc0200fff PFN 100e00

I think it is this line if I understand correctly?

0xffffffffc0200000-0xffffffffc0400000 2M ro PSE GLB x pmd

Yeah, that does seems to show it getting fixed up. Thanks! That means that the warning (at least in early boot) is essentially a false positive. The problem is that there's module-load-time patching too and that is much later and is arguably an actual problem.

Oh, and thinking about it a bit more... This warning came from alternatives patching which is too early for normal text_poke(), thus the existence of text_poke_early().

Anyway, I'm curious what Peter wants to do with this.

#54 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

Quoted 41 lines · Dave Hansen
Quoted 5 lines · Nathan Chancellor
Quoted 4 lines · Dave Hansen
Quoted 3 lines · Nathan Chancellor

I just bisected the following warning on a couple of my test machines to commit 453e78594434 ("x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()") in next-20260904.

Quoted 8 lines
Quoted 5 lines

...

Quoted 2 lines

If there is any information I can provide to help debug this, I am happy to provide it.

Well, the thing you bisected to is the patch that fixes the warning. The

Hmmm, I don't see a warning prior to this patch though. Do you mean that this patch is just revealing the bad behavior that you mention below?

Yeah, "fixes" was a bad word to use.

The W^X detection has been broken for a while. The patch you bisected to fixes the W^X detection. It doesn't actually cause the issue.

Quoted 13 lines
Quoted 5 lines

actual issue comes from:

        if (pages == &its_pages)
                set_memory_x((unsigned long)page, 1);
Quoted 7 lines

which its_alloc() does on an execmem allocation. Any chance you could get the output of:

/sys/kernel/debug/page_tables/current_kernel

for the page that page that it's complaining about? I'm curious if this is temporary or permanent.

Quoted 7 lines

Sure. For

   CPA detected W^X violation: 8000000000000123 -> 0000000000000123 range: 0xffffffffc0200000 - 0xffffffffc0200fff PFN 100e00

I think it is this line if I understand correctly?

0xffffffffc0200000-0xffffffffc0400000 2M ro PSE GLB x pmd

Quoted 8 lines

Yeah, that does seems to show it getting fixed up. Thanks! That means that the warning (at least in early boot) is essentially a false positive. The problem is that there's module-load-time patching too and that is much later and is arguably an actual problem.

Oh, and thinking about it a bit more... This warning came from alternatives patching which is too early for normal text_poke(), thus the existence of text_poke_early().

But text_poke_early is a memcpy :)

The warning at core text patching is completely spurious because anyway the entire kernel text is mapped as RWX at that point.

This one should be fixed as with the patch below. As for the modules, text_poke()ing sounds like the way to move forward and ...

Quoted 1 line

Anyway, I'm curious what Peter wants to do with this.

... I'm also curious what Peter will suggest :)

arch/x86/mm/pat/set_memory.c +4 -0
index c38faf39ce152..42ea8b4234b82 100644
@@ -686,6 +686,10 @@ static inline pgprot_t verify_rwx(pgprot_t old, pgprot_t new, unsigned long star
 	if (!(__supported_pte_mask & _PAGE_NX))
 		return new;
 
+	/* skip verification until kernel text is set to read only */
+	if (!kernel_set_to_readonly)
+		return new;
+
 	if (!((pgprot_val(old) ^ pgprot_val(new)) & (_PAGE_RW | _PAGE_NX)))
 		return new;
 
Signature · 2 lines
Sincerely yours,
Mike.
#55 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

Quoted 51 lines · Mike Rapoport
Quoted 41 lines · Dave Hansen
Quoted 5 lines · Nathan Chancellor
Quoted 4 lines · Dave Hansen
Quoted 3 lines · Nathan Chancellor

I just bisected the following warning on a couple of my test machines to commit 453e78594434 ("x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()") in next-20260904.

Quoted 8 lines
Quoted 5 lines

...

Quoted 2 lines

If there is any information I can provide to help debug this, I am happy to provide it.

Well, the thing you bisected to is the patch that fixes the warning. The

Hmmm, I don't see a warning prior to this patch though. Do you mean that this patch is just revealing the bad behavior that you mention below?

Yeah, "fixes" was a bad word to use.

The W^X detection has been broken for a while. The patch you bisected to fixes the W^X detection. It doesn't actually cause the issue.

Quoted 13 lines
Quoted 5 lines

actual issue comes from:

        if (pages == &its_pages)
                set_memory_x((unsigned long)page, 1);
Quoted 7 lines

which its_alloc() does on an execmem allocation. Any chance you could get the output of:

/sys/kernel/debug/page_tables/current_kernel

for the page that page that it's complaining about? I'm curious if this is temporary or permanent.

Quoted 7 lines

Sure. For

   CPA detected W^X violation: 8000000000000123 -> 0000000000000123 range: 0xffffffffc0200000 - 0xffffffffc0200fff PFN 100e00

I think it is this line if I understand correctly?

0xffffffffc0200000-0xffffffffc0400000 2M ro PSE GLB x pmd

Quoted 8 lines

Yeah, that does seems to show it getting fixed up. Thanks! That means that the warning (at least in early boot) is essentially a false positive. The problem is that there's module-load-time patching too and that is much later and is arguably an actual problem.

Oh, and thinking about it a bit more... This warning came from alternatives patching which is too early for normal text_poke(), thus the existence of text_poke_early().

Quoted 7 lines

But text_poke_early is a memcpy :)

The warning at core text patching is completely spurious because anyway the entire kernel text is mapped as RWX at that point.

This one should be fixed as with the patch below. As for the modules, text_poke()ing sounds like the way to move forward and ...

For modules everything should just work, the memory is RW until its_fini_mod() that resets to to ROX, so there are no RWX violations.

Looking at the code after morning coffee rather than near midnight helps :)

Signature · 2 lines
Sincerely yours,
Mike.
#56 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

In reply to Dave Hansen
Quoted 8 lines · Dave Hansen
Quoted 2 lines · Nathan Chancellor

Hmmm, I don't see a warning prior to this patch though. Do you mean that this patch is just revealing the bad behavior that you mention below?

Yeah, "fixes" was a bad word to use.

The W^X detection has been broken for a while. The patch you bisected to fixes the W^X detection. It doesn't actually cause the issue.

Ah okay, that parses way better. I assume this will be sorted out without further information needed from me but I am happy to test a patch when available.

Signature · 2 lines
Cheers,
Nathan
#57 lore raw

Re: [tip: x86/urgent] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

Hi Nathan,

Quoted 5 lines · Nathan Chancellor
Quoted 4 lines · Dave Hansen
Quoted 2 lines · Nathan Chancellor

Hmmm, I don't see a warning prior to this patch though. Do you mean that this patch is just revealing the bad behavior that you mention below?

Quoted 8 lines
Quoted 4 lines

Yeah, "fixes" was a bad word to use.

The W^X detection has been broken for a while. The patch you bisected to fixes the W^X detection. It doesn't actually cause the issue.

Ah okay, that parses way better. I assume this will be sorted out without further information needed from me but I am happy to test a patch when available.

Here:

https://lore.kernel.org/all/[email protected]/T/#u

Quoted 3 lines
Signature · 2 lines
Cheers,
Nathan
Signature · 2 lines
Sincerely yours,
Mike.
#58 lore raw

[tip: x86/mm] x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

In reply to Mike Rapoport (Microsoft)
The following commit has been merged into the x86/mm branch of tip:

The following commit has been merged into the x86/mm branch of tip:

Commit-ID:     ebaf7c9bbb1e16523d7036e7e1225ad2dfcc6482
Gitweb:        https://git.kernel.org/tip/ebaf7c9bbb1e16523d7036e7e1225ad2dfcc6482
Author:        Mike Rapoport (Microsoft) <[email protected]>
AuthorDate:    Thu, 13 Aug 2026 12:01:28 +03:00
Committer:     Dave Hansen <[email protected]>
CommitterDate: Tue, 08 Sep 2026 15:41:32 -07:00

x86/mm/pat: Fix effective RW computation in lookup_address_in_pgd_attr()

lookup_address_in_pgd_attr() accumulates the effective NX and RW bits of the walked page table levels so that verify_rwx() can detect mappings that are both writable and executable.

The RW bits are folded into a bool with

	rw &= pXd_flags(*pXd) & _PAGE_RW;

but _PAGE_RW is 0x2. So consider the accumulation line:

        rw &= pXd_flags(*pXd) & _PAGE_RW;

where rw=0x1 and the right side evaluates down to 0x2. It'll end up doing:

        rw = 0x1 & 0x2

and rw always ends up 0.

This way rw becomes false at the first level walked, regardless of the actual permissions, and verify_rwx() treats every mapping as non-writable and never reports a W^X violation.

Add double negation to the right side to normalize the _PAGE_RW flag to 0 or 1.

---
 arch/x86/mm/pat/set_memory.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/mm/pat/set_memory.c +4 -4
index c38faf3..b306000 100644
@@ -732,7 +732,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_512G;
 	*nx |= pgd_flags(*pgd) & _PAGE_NX;
-	*rw &= pgd_flags(*pgd) & _PAGE_RW;
+	*rw &= !!(pgd_flags(*pgd) & _PAGE_RW);
 
 	p4d = p4d_offset(pgd, address);
 	if (p4d_none(*p4d))
@@ -743,7 +743,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_1G;
 	*nx |= p4d_flags(*p4d) & _PAGE_NX;
-	*rw &= p4d_flags(*p4d) & _PAGE_RW;
+	*rw &= !!(p4d_flags(*p4d) & _PAGE_RW);
 
 	pud = pud_offset(p4d, address);
 	if (pud_none(*pud))
@@ -754,7 +754,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_2M;
 	*nx |= pud_flags(*pud) & _PAGE_NX;
-	*rw &= pud_flags(*pud) & _PAGE_RW;
+	*rw &= !!(pud_flags(*pud) & _PAGE_RW);
 
 	pmd = pmd_offset(pud, address);
 	if (pmd_none(*pmd))
@@ -765,7 +765,7 @@ pte_t *lookup_address_in_pgd_attr(pgd_t *pgd, unsigned long address,
 
 	*level = PG_LEVEL_4K;
 	*nx |= pmd_flags(*pmd) & _PAGE_NX;
-	*rw &= pmd_flags(*pmd) & _PAGE_RW;
+	*rw &= !!(pmd_flags(*pmd) & _PAGE_RW);
 
 	return pte_offset_kernel(pmd, address);
 }
Tested-by on the v2 series
#59 lore raw
In reply to Mike Rapoport
Quoted 45 lines · Mike Rapoport

The first three patches are urgent, the third patch fixes BUG() reported y several people and it depends on the first two.

There were no bug reports that the last two patches fix because bug manifestations won't yell at users.

TL;DR version:

There are a couple of CPA fixes floating around:

Denis Lunev fixed races between split and collapse of the large mappings:

https://lore.kernel.org/all/[email protected]

Lorenzo Stoakes fixed UAF caused by races between CPA and ptdump:

https://lore.kernel.org/all/[email protected]

and an issue with stale page tables in IOMMU:

https://lore.kernel.org/all/[email protected]

Mike Rapoport fixed a check of RW attribute in lookup_address_in_pgd_attr() used for the verification of RWX:

https://lore.kernel.org/all/[email protected]

Pedro Falcato closed a race between text poking and collapse of large pages:

https://lore.kernel.org/all/anCK3eWFMwZqq5ka@pedro-suse

Some of the fixes got merged into x86 tree, some of them got merged into mm tree and some are still hanging in the air.

The changes here are collected from all these fixes into a single coherent set on top of tip/x86/mm:

* fix for races between CPA and ptdump causing UAF
* update to the fix of the race between split and collapse of large mappings
* fix for races between CPA and vmalloc_to_page() in text poking
* fix for stale page tables in IOMMU
* fix for effective RW computation in lookup_address_in_pgd_attr()

Quoted 5 lines

---
v2 changes:
* rebased on the current tip/x86/mm that includes peterz's changes for DEBUG_PAGEALLOC
* added fix for CPA vs text poking race

Tested v2 on a 512-guest concurrent boot harness across 200 iterations with no splats. Both crash signatures reported in [1] are no longer seen.

Regards,
Nikunj

[1] https://lore.kernel.org/all/[email protected]/

Quoted 26 lines

v1: https://patch.msgid.link/[email protected]

---
Lorenzo Stoakes (ARM) (3):
x86/mm/pat: acquire init_mm write lock on collapse to avoid UAF x86/mm/pat: acquire init_mm read lock on attribute change to avoid UAF x86/mm/pat: allocate split page tables as kernel page tables

Mike Rapoport (Microsoft) (1):
x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()

Pedro Falcato (1):
x86/alternative: exclude text poking against change_page_attr()

 arch/x86/kernel/alternative.c | 39 ++++++++++++++++++++++++---
 arch/x86/mm/pat/set_memory.c  | 61 +++++++++++++++++++++++++++++++------------
 include/linux/mmap_lock.h     |  2 ++
 3 files changed, 83 insertions(+), 19 deletions(-)

---
base-commit: 7da514d819a0afb148634aac92b3d190f34947c3
change-id: 20260727-cpa-fixes-d3c73c075672

--
Sincerely yours,
Mike.

#60 lore raw
Quoted 5 lines · Nikunj A. Dadhania

Tested v2 on a 512-guest concurrent boot harness across 200 iterations with no splats. Both crash signatures reported in [1] are no longer seen.

Thanks so much for the testing!

And all credit to Pedro for reporting and contributing the key commit that resolves this issue :)

Quoted 33 lines

Regards,
Nikunj

[1] https://lore.kernel.org/all/[email protected]/

Quoted 26 lines

v1: https://patch.msgid.link/[email protected]

---
Lorenzo Stoakes (ARM) (3):
x86/mm/pat: acquire init_mm write lock on collapse to avoid UAF x86/mm/pat: acquire init_mm read lock on attribute change to avoid UAF x86/mm/pat: allocate split page tables as kernel page tables

Mike Rapoport (Microsoft) (1):
x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()

Pedro Falcato (1):
x86/alternative: exclude text poking against change_page_attr()

 arch/x86/kernel/alternative.c | 39 ++++++++++++++++++++++++---
 arch/x86/mm/pat/set_memory.c  | 61 +++++++++++++++++++++++++++++++------------
 include/linux/mmap_lock.h     |  2 ++
 3 files changed, 83 insertions(+), 19 deletions(-)

---
base-commit: 7da514d819a0afb148634aac92b3d190f34947c3
change-id: 20260727-cpa-fixes-d3c73c075672

--
Sincerely yours,
Mike.

--
Cheers, Lorenzo

#61 lore raw
Quoted 6 lines · Lorenzo Stoakes (ARM)
Quoted 5 lines · Nikunj A. Dadhania

Tested v2 on a 512-guest concurrent boot harness across 200 iterations with no splats. Both crash signatures reported in [1] are no longer seen.

Nice!

Quoted 5 lines

Thanks so much for the testing!

And all credit to Pedro for reporting and contributing the key commit that resolves this issue :)

And all credit to you for starting this flurry of fixes and patches that solve Critical Kernel Issues(tm)! My fix would be nothing without yours!

Signature · 1 line
Pedro
maintainer
#62 lore raw
In reply to Mike Rapoport
Quoted 2 lines · On Thu, 13 Aug 2026 12:01:23 +0300 Mike Rapoport <[email protected]> wrote:

Some of the fixes got merged into x86 tree, some of them got merged into mm tree and some are still hanging in the air.

I'm assuming/hoping that this whole series will be handled by the x86 team.

I have retained three random pat patches in mm.git since July 23:

x86-mm-pat-acquire-init_mm-write-lock-on-collapse-to-avoid-uaf.patch x86-mm-pat-acquire-init_mm-read-lock-on-attribute-change-to-avoid-uaf.patch x86-mm-pat-allocate-split-page-tables-as-kernel-page-tables.patch

just to get them some testing exposure. But they're old. I've heard no reports since adding them.

There's nothing more to be learned by keeping these in mm.git so I'll remove them now.

Tested-by on the v2 series
#63 lore raw
In reply to Mike Rapoport
Quoted 65 lines · Mike Rapoport

The first three patches are urgent, the third patch fixes BUG() reported y several people and it depends on the first two.

There were no bug reports that the last two patches fix because bug manifestations won't yell at users.

TL;DR version:

There are a couple of CPA fixes floating around:

Denis Lunev fixed races between split and collapse of the large mappings:

https://lore.kernel.org/all/[email protected]

Lorenzo Stoakes fixed UAF caused by races between CPA and ptdump:

https://lore.kernel.org/all/[email protected]

and an issue with stale page tables in IOMMU:

https://lore.kernel.org/all/[email protected]

Mike Rapoport fixed a check of RW attribute in lookup_address_in_pgd_attr() used for the verification of RWX:

https://lore.kernel.org/all/[email protected]

Pedro Falcato closed a race between text poking and collapse of large pages:

https://lore.kernel.org/all/anCK3eWFMwZqq5ka@pedro-suse

Some of the fixes got merged into x86 tree, some of them got merged into mm tree and some are still hanging in the air.

The changes here are collected from all these fixes into a single coherent set on top of tip/x86/mm:

* fix for races between CPA and ptdump causing UAF
* update to the fix of the race between split and collapse of large mappings
* fix for races between CPA and vmalloc_to_page() in text poking
* fix for stale page tables in IOMMU
* fix for effective RW computation in lookup_address_in_pgd_attr()

---
v2 changes:
* rebased on the current tip/x86/mm that includes peterz's changes for DEBUG_PAGEALLOC
* added fix for CPA vs text poking race

v1: https://patch.msgid.link/[email protected]

---
Lorenzo Stoakes (ARM) (3):
x86/mm/pat: acquire init_mm write lock on collapse to avoid UAF x86/mm/pat: acquire init_mm read lock on attribute change to avoid UAF x86/mm/pat: allocate split page tables as kernel page tables

Mike Rapoport (Microsoft) (1):
x86/mm/pat: fix effective RW computation in lookup_address_in_pgd_attr()

Pedro Falcato (1):
x86/alternative: exclude text poking against change_page_attr()

Quoted 7 lines
  arch/x86/kernel/alternative.c | 39 ++++++++++++++++++++++++---
  arch/x86/mm/pat/set_memory.c  | 61 +++++++++++++++++++++++++++++++------------
  include/linux/mmap_lock.h     |  2 ++
  3 files changed, 83 insertions(+), 19 deletions(-)

---
base-commit: 7da514d819a0afb148634aac92b3d190f34947c3
change-id: 20260727-cpa-fixes-d3c73c075672

Reproduced and verified this series (patches 1-3) on 4vcpu guest running two different kernels
1. mainline (commit: 77ae27fd98f3)
2. Ubuntu Resolute 7.0.0-26 (production kernel hitting the issue in a VM)

The Reproducer consisted of
1. A debug patch a cmdline-gated stall between the two *pmd reads in vmalloc_to_page()
2. One taskset-pinned insmod/rmmod worker per module over stock cfg80211/dummy/veth modules

With the above reproducer, both BUG within seconds in unpatched kernel.
1. Resolute in 0.71s at alternative.c:2564 (BUG_ON(!pages[0] ...), RAX=0)
2. mainline at alternative.c:2473 (BUG_ON(memcmp(addr, src, len))).

With patches 1-3: zero splats across 10 runs each (600s/11,378 module load/unload cycles on mainline and 1200s/55,980 module load/unload cycles on Resolute).

Quoted 4 lines

--
Sincerely yours,
Mike.

#64 lore raw
Quoted 8 lines · Atish Patra

Reproduced and verified this series (patches 1-3) on 4vcpu guest running two different kernels
1. mainline (commit: 77ae27fd98f3)
2. Ubuntu Resolute 7.0.0-26 (production kernel hitting the issue in a VM)

The Reproducer consisted of
1. A debug patch a cmdline-gated stall between the two *pmd reads in vmalloc_to_page()
2. One taskset-pinned insmod/rmmod worker per module over stock cfg80211/dummy/veth modules

Quoted 5 lines

With the above reproducer, both BUG within seconds in unpatched kernel.
1. Resolute in 0.71s at alternative.c:2564 (BUG_ON(!pages[0] ...), RAX=0)
2. mainline at alternative.c:2473 (BUG_ON(memcmp(addr, src, len))).

With patches 1-3: zero splats across 10 runs each (600s/11,378 module load/unload cycles on mainline and 1200s/55,980 module load/unload cycles on Resolute).

Amazing, thanks!

Quoted 3 lines

I guess should be applied to patches 1-3 only strictly? Or perhaps 3/5?

Quoted 5 lines
Quoted 5 lines

--
Sincerely yours,
Mike.

--
Cheers, Lorenzo

#65 lore raw
Quoted 13 lines · Lorenzo Stoakes (ARM)
Quoted 12 lines · Atish Patra

Reproduced and verified this series (patches 1-3) on 4vcpu guest running two different kernels
1. mainline (commit: 77ae27fd98f3)
2. Ubuntu Resolute 7.0.0-26 (production kernel hitting the issue in a VM)

The Reproducer consisted of
1. A debug patch a cmdline-gated stall between the two *pmd reads in vmalloc_to_page()
2. One taskset-pinned insmod/rmmod worker per module over stock cfg80211/dummy/veth modules

With the above reproducer, both BUG within seconds in unpatched kernel.
1. Resolute in 0.71s at alternative.c:2564 (BUG_ON(!pages[0] ...), RAX=0)
2. mainline at alternative.c:2473 (BUG_ON(memcmp(addr, src, len))).

Quoted 6 lines
Quoted 1 line

With patches 1-3: zero splats across 10 runs each (600s/11,378 module load/unload cycles on mainline and 1200s/55,980 module load/unload cycles on Resolute).

Amazing, thanks!

Quoted 2 lines

I guess should be applied to patches 1-3 only strictly? Or perhaps 3/5?

Yes. 1-3 sounds good.

Quoted 6 lines
Quoted 4 lines
Quoted 4 lines

--
Sincerely yours,
Mike.

--
Cheers, Lorenzo