fork: clear p->mm when copy_process() fails
Subsystems (6)
David Hildenbrand (Arm)
maintainerPATCH 1/1 fork: clear p->mm when copy_process() fails
From: Chris Mason <[email protected]>
When copy_process() fails after copy_mm(), the error path drops the mm reference but leaves p->mm set. Perf and BPF can still reach the task, and take a reference on the already freed mm through get_task_mm() or bpf_find_vma(), resulting in a use-after-free:
copy_process() PERF_EVENT_IOC_SET_FILTER
============== =========================
perf_event_init_task(p)
copy_mm(p)
a later step fails
mmput(p->mm)
/* frees the mm */
perf_event_addr_filters_apply()
get_task_mm(p)
/* use-after-free */Similarly, a BPF program on the task_free LSM hook can call bpf_find_vma(p) before the task is freed.
Fix it by clearing p->mm under task_lock() before dropping the reference, just like exit_mm() does.
Further, if copy_process() fails before copy_mm(), p->mm is still the parent's mm pointer copied by dup_task_struct(), without having taken a reference. While nobody can access it concurrently, there is the chance that it might get accessed later when a BPF program is attached to free_task().
With CONFIG_MEMCG the task is freed after an RCU grace period, so the parent might already be gone and p->mm in the child points at something that was already freed. So let's just clear p->mm before freeing the task. No lock is required as there cannot be concurrent access (and we might not even have the lock initialized yet).
This has medium urgency.
---
I can take this through my tree (rather soonish) unless someone else wants to pick this up.
--- kernel/fork.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)
kernel/fork.c +13 -2
@@ -2628,8 +2628,14 @@ __latent_entropy struct task_struct *copy_process( bad_fork_cleanup_mm:sched_cache_fork_cleanup (p);if (p->mm) { -mm_clear_owner (p->mm, p); -mmput (p->mm); +struct mm_struct *mm = p->mm; + +/* Perf events and BPF programs can still reach p. */ +task_lock (p); + p->mm =NULL ; +task_unlock (p); +mm_clear_owner (mm, p); +mmput (mm); } bad_fork_cleanup_signal:if (!(clone_flags &CLONE_THREAD ))
@@ -2663,6 +2669,11 @@ __latent_entropy struct task_struct *copy_process(exit_cred_namespaces (p);exit_creds (p); bad_fork_free: +/* +* If set, p->mm points at the parent's mm without a reference. Nobody +* can access it concurrently anymore, but prevent later access. +*/ + p->mm =NULL ;WRITE_ONCE (p->__state,TASK_DEAD );exit_task_stack_account (p);put_task_stack (p);
---
base-commit: 4405c18c95d2d0f99f6c581ef0ee8591a1aa6ccd
change-id: 20261119-mm_failed_fork-a46141b503d0
Best regards,
Signature · 1 line
David Hildenbrand (Arm) <[email protected]>
Quoted 38 lines · David Hildenbrand
From: Chris Mason <[email protected]>
When
copy_process()fails aftercopy_mm(), the error path drops the mm reference but leavesp->mmset. Perf and BPF can still reach the task, and take a reference on the already freed mm throughget_task_mm()orbpf_find_vma(), resulting in a use-after-free:copy_process() PERF_EVENT_IOC_SET_FILTER ============== ========================= perf_event_init_task(p) copy_mm(p) a later step fails mmput(p->mm) /* frees the mm */ perf_event_addr_filters_apply() get_task_mm(p) /* use-after-free */Similarly, a BPF program on the
task_freeLSM hook can callbpf_find_vma(p)before the task is freed.Fix it by clearing
p->mmundertask_lock()before dropping the reference, just likeexit_mm()does.Further, if
copy_process()fails beforecopy_mm(),p->mmis still the parent's mm pointer copied bydup_task_struct(), without having taken a reference. While nobody can access it concurrently, there is the chance that it might get accessed later when a BPF program is attached tofree_task().With
CONFIG_MEMCGthe task is freed after an RCU grace period, so the parent might already be gone andp->mmin the child points at something that was already freed. So let's just clearp->mmbefore freeing the task. No lock is required as there cannot be concurrent access (and we might not even have the lock initialized yet).This has medium urgency.
Quoted 8 lines
Fixes 375637bc5249 ("perf/core: Introduce address range filtering")Fixes 7c7e3d31e785 ("bpf: Introduce helper bpf_find_vma")Reported-by Sashiko <[email protected]>Signed-off-by Chris Mason <[email protected]>[ david: simplify description and comments ]Signed-off-by David Hildenbrand (Arm) <[email protected]>
Thanks!
Quoted 48 lines
---
I can take this through my tree (rather soonish) unless someone else wants to pick this up.--- kernel/fork.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)kernel/fork.c +13
-3@@ -2628,8 +2628,14 @@ __latent_entropy struct task_struct *copy_process( bad_fork_cleanup_mm:sched_cache_fork_cleanup (p);if (p->mm) { -mm_clear_owner (p->mm, p); -mmput (p->mm); +struct mm_struct *mm = p->mm; + +/* Perf events and BPF programs can still reach p. */ +task_lock (p); + p->mm =NULL ; +task_unlock (p); +mm_clear_owner (mm, p); +mmput (mm); } bad_fork_cleanup_signal:if (!(clone_flags &CLONE_THREAD ))@@ -2663,6 +2669,11 @@ __latent_entropy struct task_struct *copy_process(exit_cred_namespaces (p);exit_creds (p); bad_fork_free: +/* +* If set, p->mm points at the parent's mm without a reference. Nobody +* can access it concurrently anymore, but prevent later access. +*/ + p->mm =NULL ;WRITE_ONCE (p->__state,TASK_DEAD );exit_task_stack_account (p);put_task_stack (p); ---base-commit:
4405c18c95d2d0f99f6c581ef0ee8591a1aa6ccd
change-id: 20261119-mm_failed_fork-a46141b503d0Best regards,
Signature · 1 line
David Hildenbrand (Arm) <[email protected]>
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 6 lines · David Hildenbrand (Arm)
From: Chris Mason <[email protected]>
When
copy_process()fails aftercopy_mm(), the error path drops the mm reference but leavesp->mmset. Perf and BPF can still reach the task, and take a reference on the already freed mm throughget_task_mm()orbpf_find_vma(), resulting in a use-after-free:
Good lord :)
Quoted 12 lines
copy_process() PERF_EVENT_IOC_SET_FILTER
============== =========================
perf_event_init_task(p)
copy_mm(p)
a later step fails
mmput(p->mm)
/* frees the mm */
perf_event_addr_filters_apply()
get_task_mm(p)
/* use-after-free */Quoted 5 lines
Similarly, a BPF program on the
task_freeLSM hook can callbpf_find_vma(p)before the task is freed.Fix it by clearing
p->mmundertask_lock()before dropping the reference, just likeexit_mm()does.
Sounds sensible.
Quoted 7 lines
Further, if
copy_process()fails beforecopy_mm(),p->mmis still the parent's mm pointer copied bydup_task_struct(), without having taken a reference. While nobody can access it concurrently, there is the chance that it might get accessed later when a BPF program is attached tofree_task().
Quoted 7 lines
With
CONFIG_MEMCGthe task is freed after an RCU grace period, so the parent might already be gone andp->mmin the child points at something that was already freed. So let's just clearp->mmbefore freeing the task. No lock is required as there cannot be concurrent access (and we might not even have the lock initialized yet).This has medium urgency.
It sounds... horrendous though :) but I guess possibly hard to hit?
Quoted 9 lines
Fixes 375637bc5249 ("perf/core: Introduce address range filtering")Fixes 7c7e3d31e785 ("bpf: Introduce helper bpf_find_vma")Reported-by Sashiko <[email protected]>Signed-off-by Chris Mason <[email protected]>[ david: simplify description and comments ]Signed-off-by David Hildenbrand (Arm) <[email protected]>
LGTM so:
Quoted 45 lines
---
I can take this through my tree (rather soonish) unless someone else wants to pick this up.--- kernel/fork.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)kernel/fork.c +13
-3@@ -2628,8 +2628,14 @@ __latent_entropy struct task_struct *copy_process( bad_fork_cleanup_mm:sched_cache_fork_cleanup (p);if (p->mm) { -mm_clear_owner (p->mm, p); -mmput (p->mm); +struct mm_struct *mm = p->mm; + +/* Perf events and BPF programs can still reach p. */ +task_lock (p); + p->mm =NULL ; +task_unlock (p); +mm_clear_owner (mm, p); +mmput (mm); } bad_fork_cleanup_signal:if (!(clone_flags &CLONE_THREAD ))@@ -2663,6 +2669,11 @@ __latent_entropy struct task_struct *copy_process(exit_cred_namespaces (p);exit_creds (p); bad_fork_free: +/* +* If set, p->mm points at the parent's mm without a reference. Nobody +* can access it concurrently anymore, but prevent later access. +*/ + p->mm =NULL ;WRITE_ONCE (p->__state,TASK_DEAD );exit_task_stack_account (p);put_task_stack (p); ---base-commit:
4405c18c95d2d0f99f6c581ef0ee8591a1aa6ccd
change-id: 20261119-mm_failed_fork-a46141b503d0
Quoted 4 lines
Best regards,
--
David Hildenbrand (Arm) <[email protected]>
--
Cheers, Lorenzo
Quoted 39 lines · David Hildenbrand (Arm)
From: Chris Mason <[email protected]>
When
copy_process()fails aftercopy_mm(), the error path drops the mm reference but leavesp->mmset. Perf and BPF can still reach the task, and take a reference on the already freed mm throughget_task_mm()orbpf_find_vma(), resulting in a use-after-free:copy_process() PERF_EVENT_IOC_SET_FILTER ============== ========================= perf_event_init_task(p) copy_mm(p) a later step fails mmput(p->mm) /* frees the mm */ perf_event_addr_filters_apply() get_task_mm(p) /* use-after-free */Similarly, a BPF program on the
task_freeLSM hook can callbpf_find_vma(p)
before the task is freed.Fix it by clearing
p->mmundertask_lock()before dropping the reference, just likeexit_mm()does.Further, if
copy_process()fails beforecopy_mm(),p->mmis still the parent's mm pointer copied bydup_task_struct(), without having taken a reference. While nobody can access it concurrently, there is the chance that it might get accessed later when a BPF program is attached tofree_task().With
CONFIG_MEMCGthe task is freed after an RCU grace period, so the parent might already be gone andp->mmin the child points at something that was already freed. So let's just clearp->mmbefore freeing the task. No lock is required as there cannot be concurrent access (and we might not even have the lock initialized yet).This has medium urgency.
Quoted 7 lines
Fixes 375637bc5249 ("perf/core: Introduce address range filtering")Fixes 7c7e3d31e785 ("bpf: Introduce helper bpf_find_vma")Reported-by Sashiko <[email protected]>Signed-off-by Chris Mason <[email protected]>[ david: simplify description and comments ]
Quoted 42 lines
Signed-off-by David Hildenbrand (Arm) <[email protected]>---
I can take this through my tree (rather soonish) unless someone else wants to pick this up.
---kernel/fork.c| 15 +++++++++++++--1 file changed, 13 insertions(+), 2 deletions(-)kernel/fork.c +0
-0@@ -2628,8 +2628,14 @@ __latent_entropy struct task_struct *copy_process(
bad_fork_cleanup_mm:sched_cache_fork_cleanup(p); if (p->mm) {kernel/fork.c +8
-2-mm_clear_owner (p->mm, p); -mmput (p->mm); +struct mm_struct *mm = p->mm; + +/* Perf events and BPF programs can still reach p. */ +task_lock (p); + p->mm =NULL ; +task_unlock (p); +mm_clear_owner (mm, p); +mmput (mm);}
bad_fork_cleanup_signal:
if (!(clone_flags &CLONE_THREAD))kernel/fork.c +0
-0@@ -2663,6 +2669,11 @@ __latent_entropy struct task_struct *copy_process(exit_cred_namespaces (p);exit_creds (p);
bad_fork_free:kernel/fork.c +5
-0+/* +* If set, p->mm points at the parent's mm without a reference. Nobody +* can access it concurrently anymore, but prevent later access. +*/ + p->mm =NULL ;WRITE_ONCE(p->__state, TASK_DEAD); exit_task_stack_account(p); put_task_stack(p);
Quoted 10 lines
---
base-commit:4405c18c95d2d0f99f6c581ef0ee8591a1aa6ccd
change-id: 20261119-mm_failed_fork-a46141b503d0Best regards,
Signature · 1 line
David Hildenbrand (Arm) <[email protected]>
--- Thanks!
Quoted 16 lines · David Hildenbrand (Arm)
When
copy_process()fails aftercopy_mm(), the error path drops the mm reference but leavesp->mmset. Perf and BPF can still reach the task, and take a reference on the already freed mm throughget_task_mm()orbpf_find_vma(), resulting in a use-after-free:copy_process() PERF_EVENT_IOC_SET_FILTER ============== ========================= perf_event_init_task(p) copy_mm(p) a later step fails mmput(p->mm) /* frees the mm */ perf_event_addr_filters_apply() get_task_mm(p) /* use-after-free */
Quoted 1 line
[...]
Applied, thanks!
tree: mm/core.git
branch: for-next-fixes
Applied patches will appear soon in mm-next, followed by linux-next.
Please report any outstanding bugs that were missed during review, so we can either drop the patch series or decide how to fix it up.
It is encouraged to provide Acked-bys, Reviewed-bys, and Tested-bys even though the patches have already been applied. If possible, patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase, trailer updates or similar. Further, this is not a guarantee that the patches will go upstream. If in doubt, please check the listed branch.
[1/1] fork: clear p->mm when copy_process() fails
https://git.kernel.org/mm/core/c/c9b8c9bd
Signature · 3 lines
Cheers, David
Quoted 46 lines · David Hildenbrand (Arm)
From: Chris Mason <[email protected]>
When
copy_process()fails aftercopy_mm(), the error path drops the mm reference but leavesp->mmset. Perf and BPF can still reach the task, and take a reference on the already freed mm throughget_task_mm()orbpf_find_vma(), resulting in a use-after-free:copy_process() PERF_EVENT_IOC_SET_FILTER ============== ========================= perf_event_init_task(p) copy_mm(p) a later step fails mmput(p->mm) /* frees the mm */ perf_event_addr_filters_apply() get_task_mm(p) /* use-after-free */Similarly, a BPF program on the
task_freeLSM hook can callbpf_find_vma(p)before the task is freed.Fix it by clearing
p->mmundertask_lock()before dropping the reference, just likeexit_mm()does.Further, if
copy_process()fails beforecopy_mm(),p->mmis still the parent's mm pointer copied bydup_task_struct(), without having taken a reference. While nobody can access it concurrently, there is the chance that it might get accessed later when a BPF program is attached tofree_task().With
CONFIG_MEMCGthe task is freed after an RCU grace period, so the parent might already be gone andp->mmin the child points at something that was already freed. So let's just clearp->mmbefore freeing the task. No lock is required as there cannot be concurrent access (and we might not even have the lock initialized yet).This has medium urgency.
Fixes 375637bc5249 ("perf/core: Introduce address range filtering")Fixes 7c7e3d31e785 ("bpf: Introduce helper bpf_find_vma")Reported-by Sashiko <[email protected]>Signed-off-by Chris Mason <[email protected]>[ david: simplify description and comments ]Signed-off-by David Hildenbrand (Arm) <[email protected]>
Quoted 3 lines
---
I can take this through my tree (rather soonish) unless someone else wants to pick this up.
Have at.