Skip to contentSkip to conversation

arm64/mm: Fix use-after-free due to race between memory hotunplug and ptdump

PATCH In review Dev Jain 2 participants 5 messages Jul 28, 2025

Subsystems (2)
lore
Share & tools
lore

Dev Jain

author
#0 lore raw

PATCH 1/1 arm64/mm: Fix use-after-free due to race between memory hotunplug and ptdump

Memory hotunplug is done under the hotplug lock and ptdump walk is done under the init_mm.mmap_lock. Therefore, ptdump and hotunplug can run simultaneously without any synchronization. During hotunplug, free_empty_tables() is ultimately called to free up the pagetables. The following race can happen, where x denotes the level of the pagetable:

CPU1 CPU2
free_empty_pxd_table
ptdump_walk_pgd() Get p(x+1)d table from pxd entry
pxd_clear
free_hotplug_pgtable_page(p(x+1)dp)
Still using the p(x+1)d table

which leads to a user-after-free.

To solve this, we need to synchronize ptdump_walk_pgd() with free_hotplug_pgtable_page() in such a way that ptdump never takes a reference on a freed pagetable.

Since this race is very unlikely to happen in practice, we do not want to penalize other code paths taking the init_mm mmap_lock. Therefore, we use static keys. ptdump will enable the static key - upon observing that, the free_empty_pxd_table() functions will get patched in with an mmap_read_lock/unlock sequence. A code comment explains in detail, how a combination of acquire semantics of static_branch_enable() and the barriers in __flush_tlb_kernel_pgtable() ensures that ptdump will never get a hold on the address of a freed pagetable - either ptdump will block the table freeing path due to write locking the mmap_lock, or, the nullity of the pxd entry will be observed by ptdump, therefore having no access to the isolated p(x+1)d pagetable.

This bug was found by code inspection, as a result of working on [1].
1. https://lore.kernel.org/all/[email protected]/

Fixes bbd6ec605c0f ("arm64/mm: Enable memory hot remove")

---
Rebased on Linux 6.16.

 arch/arm64/include/asm/ptdump.h |  2 ++
 arch/arm64/mm/mmu.c             | 61 +++++++++++++++++++++++++++++++++
 arch/arm64/mm/ptdump.c          | 11 ++++--
 3 files changed, 72 insertions(+), 2 deletions(-)
arch/arm64/include/asm/ptdump.h +2 -0
index fded5358641f..4760168cbd6e 100644
@@ -7,6 +7,8 @@
 
 #include <linux/ptdump.h>
 
+DECLARE_STATIC_KEY_FALSE(arm64_ptdump_key);
+
 #ifdef CONFIG_PTDUMP
 
 #include <linux/mm_types.h>
arch/arm64/mm/mmu.c +61 -0

97 lines not on this page · raw message

arch/arm64/mm/ptdump.c +9 -2
index 421a5de806c6..d543c9f8ffa8 100644
@@ -283,6 +283,13 @@ void note_page_flush(struct ptdump_state *pt_st)
 	note_page(pt_st, 0, -1, pte_val(pte_zero));
 }
 
+static void arm64_ptdump_walk_pgd(struct ptdump_state *st, struct mm_struct *mm)
+{
+	static_branch_enable(&arm64_ptdump_key);
+	ptdump_walk_pgd(st, mm, NULL);
+	static_branch_disable(&arm64_ptdump_key);
+}
+
 void ptdump_walk(struct seq_file *s, struct ptdump_info *info)
 {
 	unsigned long end = ~0UL;
@@ -311,7 +318,7 @@ void ptdump_walk(struct seq_file *s, struct ptdump_info *info)
 		}
 	};
 
-	ptdump_walk_pgd(&st.ptdump, info->mm, NULL);
+	arm64_ptdump_walk_pgd(&st.ptdump, info->mm);
 }
 
 static void __init ptdump_initialize(void)
@@ -353,7 +360,7 @@ bool ptdump_check_wx(void)
 		}
 	};
 
-	ptdump_walk_pgd(&st.ptdump, &init_mm, NULL);
+	arm64_ptdump_walk_pgd(&st.ptdump, &init_mm);
 
 	if (st.wx_pages || st.uxn_pages) {
 		pr_warn("Checked W+X mappings: FAILED, %lu W+X pages found, %lu non-UXN pages found\n",
Signature · 1 line
2.30.2
author
#1 lore raw
Quoted 32 lines · Dev Jain

Memory hotunplug is done under the hotplug lock and ptdump walk is done under the init_mm.mmap_lock. Therefore, ptdump and hotunplug can run simultaneously without any synchronization. During hotunplug, free_empty_tables() is ultimately called to free up the pagetables. The following race can happen, where x denotes the level of the pagetable:

CPU1 CPU2
free_empty_pxd_table
ptdump_walk_pgd() Get p(x+1)d table from pxd entry
pxd_clear
free_hotplug_pgtable_page(p(x+1)dp)
Still using the p(x+1)d table

which leads to a user-after-free.

To solve this, we need to synchronize ptdump_walk_pgd() with free_hotplug_pgtable_page() in such a way that ptdump never takes a reference on a freed pagetable.

Since this race is very unlikely to happen in practice, we do not want to penalize other code paths taking the init_mm mmap_lock. Therefore, we use static keys. ptdump will enable the static key - upon observing that, the free_empty_pxd_table() functions will get patched in with an mmap_read_lock/unlock sequence. A code comment explains in detail, how a combination of acquire semantics of static_branch_enable() and the barriers in __flush_tlb_kernel_pgtable() ensures that ptdump will never get a hold on the address of a freed pagetable - either ptdump will block the table freeing path due to write locking the mmap_lock, or, the nullity of the pxd entry will be observed by ptdump, therefore having no access to the isolated p(x+1)d pagetable.

Quoted 7 lines

This bug was found by code inspection, as a result of working on [1].
1. https://lore.kernel.org/all/[email protected]/

Fixes bbd6ec605c0f ("arm64/mm: Enable memory hot remove")

---

Immediately after posting, I guess the first objection which is going to come is, why not just nest free_empty_tables() with mmap_read_lock/unlock. Memory offlining obviously should not be a hot path so taking the read lock should be fine I guess.

#2 lore raw
In reply to Dev Jain
Quoted 14 lines · Dev Jain

Memory hotunplug is done under the hotplug lock and ptdump walk is done under the init_mm.mmap_lock. Therefore, ptdump and hotunplug can run simultaneously without any synchronization. During hotunplug, free_empty_tables() is ultimately called to free up the pagetables. The following race can happen, where x denotes the level of the pagetable:

CPU1 CPU2
free_empty_pxd_table
ptdump_walk_pgd() Get p(x+1)d table from pxd entry
pxd_clear
free_hotplug_pgtable_page(p(x+1)dp)
Still using the p(x+1)d table

Quoted 1 line

which leads to a user-after-free.

I'm not sure I understand this. ptdump_show() protects against this with get_online_mems()/put_online_mems(), doesn't it? There are 2 paths that call ptdump_walk_pgd(). This protects one of them. The other is ptdump_check_wx(); I thought you (or Anshuman?) had a patch in flight to fix that with [get|put]_online_mems() too?

Sorry if my memory is failing me here...

author
#3 lore raw
Quoted 15 lines · Ryan Roberts
Quoted 14 lines · Dev Jain

Memory hotunplug is done under the hotplug lock and ptdump walk is done under the init_mm.mmap_lock. Therefore, ptdump and hotunplug can run simultaneously without any synchronization. During hotunplug, free_empty_tables() is ultimately called to free up the pagetables. The following race can happen, where x denotes the level of the pagetable:

CPU1 CPU2
free_empty_pxd_table
ptdump_walk_pgd() Get p(x+1)d table from pxd entry
pxd_clear
free_hotplug_pgtable_page(p(x+1)dp)
Still using the p(x+1)d table

Quoted 8 lines
Quoted 1 line

which leads to a user-after-free.

I'm not sure I understand this. ptdump_show() protects against this with get_online_mems()/put_online_mems(), doesn't it? There are 2 paths that call ptdump_walk_pgd(). This protects one of them. The other is ptdump_check_wx(); I thought you (or Anshuman?) had a patch in flight to fix that with [get|put]_online_mems() too?

Sorry if my memory is failing me here...

Nope, I think I just had a use-after-free in my memory so I came up with this patch :) Because of the recent work with ptdump, I was so concentrated on ptdump_walk_pgd() that I didn't even bother looking up the call chain. And I even forgot we had these [get|put]_online_mems() patches recently.

Sorry for the noise, it must have been incredibly confusing to see this patch :(

#4 lore raw
Quoted 26 lines · Dev Jain
Quoted 15 lines · Ryan Roberts
Quoted 14 lines · Dev Jain

Memory hotunplug is done under the hotplug lock and ptdump walk is done under the init_mm.mmap_lock. Therefore, ptdump and hotunplug can run simultaneously without any synchronization. During hotunplug, free_empty_tables() is ultimately called to free up the pagetables. The following race can happen, where x denotes the level of the pagetable:

CPU1                    CPU2 free_empty_pxd_table
                    ptdump_walk_pgd()
                    Get p(x+1)d table from pxd entry pxd_clear free_hotplug_pgtable_page(p(x+1)dp)
                    Still using the p(x+1)d table

Quoted 8 lines
Quoted 1 line

which leads to a user-after-free.

I'm not sure I understand this. ptdump_show() protects against this with get_online_mems()/put_online_mems(), doesn't it? There are 2 paths that call ptdump_walk_pgd(). This protects one of them. The other is ptdump_check_wx(); I thought you (or Anshuman?) had a patch in flight to fix that with [get|put]_online_mems() too?

Sorry if my memory is failing me here...

Quoted 7 lines

Nope, I think I just had a use-after-free in my memory so I came up with this patch :)
Because of the recent work with ptdump, I was so concentrated on ptdump_walk_pgd() that I
didn't even bother looking up the call chain. And I even forgot we had these [get|put]_online_mems()
patches recently.

I just checked; Anshuman's fix is in mm-stable, so I guess it'll be in v6.17-rc1.

That's the patch:
https://lore.kernel.org/linux-arm-kernel/[email protected]/

Quoted 3 lines

Sorry for the noise, it must have been incredibly confusing to see this patch :(