Skip to contentSkip to conversation

bpf: Fix UAF in task local storage

PATCH Superseded KP Singh 5 participants 8 messages Jun 1, 2023

Subsystems (4)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

KP Singh

author
#0 lore raw

PATCH 1/1 bpf: Fix UAF in task local storage

When the the task local storage was generalized for tracing programs, the bpf_task_local_storage callback was moved from a BPF LSM hook callback for security_task_free LSM hook to it's own callback. But a failure case in bad_fork_cleanup_security was missed which, when triggered, led to a dangling task owner pointer and a subsequent use-after-free.

This issue was noticed when a BPF LSM program was attached to the task_alloc hook on a kernel with KASAN enabled. The program used bpf_task_storage_get to copy the task local storage from the current task to the new task being created.

Fixes a10787e6d58c ("bpf: Enable task local storage for tracing programs")

---

This fixes the regression from the LSM blob based implementation, we can still have UAFs, if bpf_task_storage_get is invoked after bpf_task_storage_free in the cleanup path.

 kernel/fork.c | 1 +
 1 file changed, 1 insertion(+)
kernel/fork.c +1 -0
index ed4e01daccaa..112d85091ae6 100644
@@ -2781,6 +2781,7 @@ __latent_entropy struct task_struct *copy_process(
 	exit_sem(p);
 bad_fork_cleanup_security:
 	security_task_free(p);
+	bpf_task_storage_free(p);
 bad_fork_cleanup_audit:
 	audit_free(p);
 bad_fork_cleanup_perf:
Signature · 1 line
2.41.0.rc0.172.g3f132b7071-goog
reviewer
#1 lore raw
Quoted 11 lines · KP Singh

When the the task local storage was generalized for tracing programs, the bpf_task_local_storage callback was moved from a BPF LSM hook callback for security_task_free LSM hook to it's own callback. But a failure case in bad_fork_cleanup_security was missed which, when triggered, led to a dangling task owner pointer and a subsequent use-after-free.

This issue was noticed when a BPF LSM program was attached to the task_alloc hook on a kernel with KASAN enabled. The program used bpf_task_storage_get to copy the task local storage from the current task to the new task being created.

This is pretty tricky. Let's add a selftest for this.

Quoted 9 lines
Fixes a10787e6d58c ("bpf: Enable task local storage for tracing programs")

---

This fixes the regression from the LSM blob based implementation, we can still have UAFs, if bpf_task_storage_get is invoked after bpf_task_storage_free in the cleanup path.

Can we fix this by calling bpf_task_storage_free() from free_task()?

Thanks,
Song

Quoted 20 lines
 kernel/fork.c | 1 +
 1 file changed, 1 insertion(+)
kernel/fork.c +1 -1
index ed4e01daccaa..112d85091ae6 100644
@@ -2781,6 +2781,7 @@ __latent_entropy struct task_struct *copy_process(
        exit_sem(p);
 bad_fork_cleanup_security:
        security_task_free(p);
+       bpf_task_storage_free(p);
 bad_fork_cleanup_audit:
        audit_free(p);
 bad_fork_cleanup_perf:
--

2.41.0.rc0.172.g3f132b7071-goog

author
#2 lore raw
Quoted 9 lines · Song Liu
Quoted 7 lines · KP Singh

When the the task local storage was generalized for tracing programs, the bpf_task_local_storage callback was moved from a BPF LSM hook callback for security_task_free LSM hook to it's own callback. But a failure case in bad_fork_cleanup_security was missed which, when triggered, led to a dangling task owner pointer and a subsequent use-after-free.

Quoted 6 lines
Quoted 4 lines

This issue was noticed when a BPF LSM program was attached to the task_alloc hook on a kernel with KASAN enabled. The program used bpf_task_storage_get to copy the task local storage from the current task to the new task being created.

This is pretty tricky. Let's add a selftest for this.

I don't have an easy repro for this (the UAF does not trigger immediately), Also I am not sure how one would test a UAF in a selftest. What actually happens is:

* We have a dangling task pointer in local storage.
* This is used sometime later which then leads to weird memory corruption errors.

Quoted 7 lines
Quoted 7 lines
Fixes a10787e6d58c ("bpf: Enable task local storage for tracing programs")

---

Quoted 5 lines
Quoted 3 lines

This fixes the regression from the LSM blob based implementation, we can still have UAFs, if bpf_task_storage_get is invoked after bpf_task_storage_free in the cleanup path.

Can we fix this by calling bpf_task_storage_free() from free_task()?

I think we can yeah. But, this is yet another deviation from how the security blob is managed (security_task_free) frees the blob that we were previously using.

Quoted 24 lines

Thanks,
Song

Quoted 20 lines
 kernel/fork.c | 1 +
 1 file changed, 1 insertion(+)
kernel/fork.c +1 -1
index ed4e01daccaa..112d85091ae6 100644
@@ -2781,6 +2781,7 @@ __latent_entropy struct task_struct *copy_process(
        exit_sem(p);
 bad_fork_cleanup_security:
        security_task_free(p);
+       bpf_task_storage_free(p);
 bad_fork_cleanup_audit:
        audit_free(p);
 bad_fork_cleanup_perf:
--

2.41.0.rc0.172.g3f132b7071-goog

#3 lore raw
Quoted 21 lines · KP Singh
Quoted 9 lines · Song Liu
Quoted 7 lines · KP Singh

When the the task local storage was generalized for tracing programs, the bpf_task_local_storage callback was moved from a BPF LSM hook callback for security_task_free LSM hook to it's own callback. But a failure case in bad_fork_cleanup_security was missed which, when triggered, led to a dangling task owner pointer and a subsequent use-after-free.

Quoted 6 lines
Quoted 4 lines

This issue was noticed when a BPF LSM program was attached to the task_alloc hook on a kernel with KASAN enabled. The program used bpf_task_storage_get to copy the task local storage from the current task to the new task being created.

This is pretty tricky. Let's add a selftest for this.

Quoted 7 lines

I don't have an easy repro for this (the UAF does not trigger immediately), Also I am not sure how one would test a UAF in a selftest. What actually happens is:

* We have a dangling task pointer in local storage.
* This is used sometime later which then leads to weird memory corruption errors.

I think we will see it easily with KASAN, no?

Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
Fixes a10787e6d58c ("bpf: Enable task local storage for tracing programs")

---

This fixes the regression from the LSM blob based implementation, we can still have UAFs, if bpf_task_storage_get is invoked after bpf_task_storage_free in the cleanup path.

Quoted 5 lines
Quoted 1 line

Can we fix this by calling bpf_task_storage_free() from free_task()?

I think we can yeah. But, this is yet another deviation from how the security blob is managed (security_task_free) frees the blob that we were previously using.

Yeah, this will make the code even more tricky.

Another idea I had is to filter on task->__state in the helper. IOW, task local storage does not work on starting or died tasks. But I am not sure whether this will make BPF_LSM less effective (not covering certain tasks).

Thanks,
Song

#4 lore raw
Quoted 9 lines · Song Liu
Quoted 9 lines · KP Singh
Quoted 3 lines · Song Liu
Quoted 1 line · KP Singh

<...>

Quoted 15 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Fixes a10787e6d58c ("bpf: Enable task local storage for tracing programs")

---

This fixes the regression from the LSM blob based implementation, we can still have UAFs, if bpf_task_storage_get is invoked after bpf_task_storage_free in the cleanup path.

Quoted 5 lines
Quoted 1 line

Can we fix this by calling bpf_task_storage_free() from free_task()?

I think we can yeah. But, this is yet another deviation from how the security blob is managed (security_task_free) frees the blob that we were previously using.

Quoted 7 lines

Yeah, this will make the code even more tricky.

Another idea I had is to filter on task->__state in the helper. IOW, task local storage does not work on starting or died tasks. But I am not sure whether this will make BPF_LSM less effective (not covering certain tasks).

I thought about this as well. But I think some use cases would want to use task local storage at task creation. At least attaching at the sched_newtask tracepoints should be fine in those cases. But at the time of sched_newtask, task->__state is still TASK_NEW.

Another idea, is it possible to allow LSM + local storage on starting/dying tasks, but restrict tracing + local storage to created tasks?

Hao

#5 lore raw
In reply to Song Liu
Quoted 25 lines · Song Liu
Quoted 25 lines · KP Singh
Quoted 9 lines · Song Liu
Quoted 7 lines · KP Singh

When the the task local storage was generalized for tracing programs, the bpf_task_local_storage callback was moved from a BPF LSM hook callback for security_task_free LSM hook to it's own callback. But a failure case in bad_fork_cleanup_security was missed which, when triggered, led to a dangling task owner pointer and a subsequent use-after-free.

Quoted 6 lines
Quoted 4 lines

This issue was noticed when a BPF LSM program was attached to the task_alloc hook on a kernel with KASAN enabled. The program used bpf_task_storage_get to copy the task local storage from the current task to the new task being created.

This is pretty tricky. Let's add a selftest for this.

I don't have an easy repro for this (the UAF does not trigger immediately), Also I am not sure how one would test a UAF in a selftest. What actually happens is:

Quoted 23 lines
Quoted 3 lines

* We have a dangling task pointer in local storage.
* This is used sometime later which then leads to weird memory corruption errors.

I think we will see it easily with KASAN, no?

Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
Fixes a10787e6d58c ("bpf: Enable task local storage for tracing programs")

---

Quoted 5 lines
Quoted 3 lines

This fixes the regression from the LSM blob based implementation, we can still have UAFs, if bpf_task_storage_get is invoked after bpf_task_storage_free in the cleanup path.

Can we fix this by calling bpf_task_storage_free() from free_task()?

I think we can yeah. But, this is yet another deviation from how the security blob is managed (security_task_free) frees the blob that we were previously using.

Does it mean doing bpf_task_storage_free() in free_task() will break some use cases? Could you explain? Doing bpf_task_storage_free() in free_task() seems to be more straight forward.

Quoted 12 lines

Yeah, this will make the code even more tricky.

Another idea I had is to filter on task->__state in the helper. IOW, task local storage does not work on starting or died tasks. But I am not sure whether this will make BPF_LSM less effective (not covering certain tasks).

Thanks,
Song

author
#6 lore raw
Quoted 27 lines · Martin KaFai Lau
Quoted 25 lines · Song Liu
Quoted 25 lines · KP Singh
Quoted 9 lines · Song Liu
Quoted 7 lines · KP Singh

When the the task local storage was generalized for tracing programs, the bpf_task_local_storage callback was moved from a BPF LSM hook callback for security_task_free LSM hook to it's own callback. But a failure case in bad_fork_cleanup_security was missed which, when triggered, led to a dangling task owner pointer and a subsequent use-after-free.

Quoted 6 lines
Quoted 4 lines

This issue was noticed when a BPF LSM program was attached to the task_alloc hook on a kernel with KASAN enabled. The program used bpf_task_storage_get to copy the task local storage from the current task to the new task being created.

This is pretty tricky. Let's add a selftest for this.

I don't have an easy repro for this (the UAF does not trigger immediately), Also I am not sure how one would test a UAF in a selftest. What actually happens is:

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

* We have a dangling task pointer in local storage.
* This is used sometime later which then leads to weird memory corruption errors.

I think we will see it easily with KASAN, no?

No, the issue only happens when copy_process fails for some reason (which one can possibly trigger with error injection / fexit) and then somehow triggers the UAF.

Even if one does manage to trigger the KASAN warning, we won't fail the selftest, so I don't see this in the selftest territory TBH. What do you have in mind?

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 14 lines
Quoted 4 lines
Quoted 8 lines
Fixes a10787e6d58c ("bpf: Enable task local storage for tracing programs")

---

This fixes the regression from the LSM blob based implementation, we can still have UAFs, if bpf_task_storage_get is invoked after bpf_task_storage_free in the cleanup path.

Can we fix this by calling bpf_task_storage_free() from free_task()?

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines

I think we can yeah. But, this is yet another deviation from how the security blob is managed (security_task_free) frees the blob that we were previously using.

Does it mean doing bpf_task_storage_free() in free_task() will break some use cases? Could you explain?
Doing bpf_task_storage_free() in free_task() seems to be more straight forward.

Superficially, I don't see any issues . All I am saying is that, before we generalized task local storage, it was allocated and freed as a security blob and now it's deviating further. Should we just consider moving security_task_free into task_free then?

Quoted 5 lines
Quoted 5 lines

Yeah, this will make the code even more tricky.

Another idea I had is to filter on task->__state in the helper. IOW,

bailing out on __state == TASK_DEAD should be reasonable.

Quoted 3 lines
Quoted 3 lines

task local storage does not work on starting or died tasks. But I am not sure whether this will make BPF_LSM less effective (not covering certain tasks).

As long as the task local storage is usable in LSM hooks like security_task_alloc it's okay

Quoted 6 lines
Quoted 5 lines

Thanks,
Song

author
#7 lore raw
Quoted 62 lines · KP Singh
Quoted 27 lines · Martin KaFai Lau
Quoted 25 lines · Song Liu
Quoted 25 lines · KP Singh
Quoted 9 lines · Song Liu
Quoted 7 lines · KP Singh

When the the task local storage was generalized for tracing programs, the bpf_task_local_storage callback was moved from a BPF LSM hook callback for security_task_free LSM hook to it's own callback. But a failure case in bad_fork_cleanup_security was missed which, when triggered, led to a dangling task owner pointer and a subsequent use-after-free.

Quoted 6 lines
Quoted 4 lines

This issue was noticed when a BPF LSM program was attached to the task_alloc hook on a kernel with KASAN enabled. The program used bpf_task_storage_get to copy the task local storage from the current task to the new task being created.

This is pretty tricky. Let's add a selftest for this.

I don't have an easy repro for this (the UAF does not trigger immediately), Also I am not sure how one would test a UAF in a selftest. What actually happens is:

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

* We have a dangling task pointer in local storage.
* This is used sometime later which then leads to weird memory corruption errors.

I think we will see it easily with KASAN, no?

No, the issue only happens when copy_process fails for some reason (which one can possibly trigger with error injection / fexit) and then somehow triggers the UAF.

Even if one does manage to trigger the KASAN warning, we won't fail the selftest, so I don't see this in the selftest territory TBH. What do you have in mind?

Quoted 19 lines
Quoted 18 lines
Quoted 18 lines
Quoted 9 lines
Quoted 9 lines
Fixes a10787e6d58c ("bpf: Enable task local storage for tracing programs")

---

Quoted 5 lines
Quoted 3 lines

This fixes the regression from the LSM blob based implementation, we can still have UAFs, if bpf_task_storage_get is invoked after bpf_task_storage_free in the cleanup path.

Can we fix this by calling bpf_task_storage_free() from free_task()?

I think we can yeah. But, this is yet another deviation from how the security blob is managed (security_task_free) frees the blob that we were previously using.

Quoted 8 lines
Quoted 3 lines

Does it mean doing bpf_task_storage_free() in free_task() will break some use cases? Could you explain?
Doing bpf_task_storage_free() in free_task() seems to be more straight forward.

Superficially, I don't see any issues . All I am saying is that, before we generalized task local storage, it was allocated and freed as a security blob and now it's deviating further. Should we just consider moving security_task_free into task_free then?

For now, I am going to move bpf_task_storage_free into free_task as it's the immediate fix and respin.

Quoted 22 lines
Quoted 6 lines
Quoted 6 lines

Yeah, this will make the code even more tricky.

Another idea I had is to filter on task->__state in the helper. IOW,

bailing out on __state == TASK_DEAD should be reasonable.

Quoted 3 lines
Quoted 3 lines

task local storage does not work on starting or died tasks. But I am not sure whether this will make BPF_LSM less effective (not covering certain tasks).

As long as the task local storage is usable in LSM hooks like security_task_alloc it's okay

Quoted 6 lines
Quoted 5 lines

Thanks,
Song