Skip to contentSkip to conversation

bpf: Do not allow deleting local storage in NMI

PATCH Superseded Amery Hung 3 participants 4 messages Mar 18 – Mar 19 BPF

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Amery Hung

author
#0 lore raw

PATCH 1/1 bpf: Do not allow deleting local storage in NMI

Currently, local storage may deadlock when deferring freeing selem or local storage through kfree_rcu(), call_rcu() or call_rcu_tasks_trace() in NMI or reentrant. kfree_rcu() and call_rcu() have always been unsafe in NMI. call_rcu_tasks_trace() only became unsafe when it switched to call_srcu(). Since deleting selem in NMI is an unlikely use case, partially mitigate it by returning error when calling from bpf_xxx_storage_delete() helpers in NMI. The check is only added to bpf_selem_unlink(), which is used by helpers and syscalls. bpf_selem_unlink_nofail() is fine as it is called during map and owner tear down that never run in NMI or reentrant.

The long-term solution requires _nolock versions of call_rcu. Another possible solution is to defer the free through irq_work, but it would grow the size of selem, which is non-ideal.

Fixes a10787e6d58c ("bpf: Enable task local storage for tracing programs")
---
 kernel/bpf/bpf_local_storage.c | 3 +++
 1 file changed, 3 insertions(+)
kernel/bpf/bpf_local_storage.c +3 -0
index 69bb2a7a71bc..cad17ca8552f 100644
@@ -393,6 +393,9 @@ int bpf_selem_unlink(struct bpf_local_storage_elem *selem)
 	unsigned long flags;
 	int err;
 
+	if (in_nmi())
+		return -EOPNOTSUPP;
+
 	if (unlikely(!selem_linked_to_storage_lockless(selem)))
 		/* selem has already been unlinked from sk */
 		return 0;
Signature · 1 line
2.52.0
maintainer Acked-by on v1 1/1
#1 lore raw
Quoted 5 lines · Amery Hung

Currently, local storage may deadlock when deferring freeing selem or local storage through kfree_rcu(), call_rcu() or call_rcu_tasks_trace() in NMI or reentrant. kfree_rcu() and call_rcu() have always been unsafe in NMI. call_rcu_tasks_trace() only became unsafe when it switched to

It has also always been unsafe in NMI, so the commit log needs an update. We should also note that the correct solution is irqs_disabled() to completely close the same context reentrancy, but it is too coarse-grained for this case, and we can revisit it once we convert to the _nolock() variant.

Quoted 7 lines

call_srcu(). Since deleting selem in NMI is an unlikely use case, partially mitigate it by returning error when calling from bpf_xxx_storage_delete() helpers in NMI. The check is only added to bpf_selem_unlink(), which is used by helpers and syscalls. bpf_selem_unlink_nofail() is fine as it is called during map and owner tear down that never run in NMI or reentrant.

Quoted 7 lines

The long-term solution requires _nolock versions of call_rcu. Another possible solution is to defer the free through irq_work, but it would grow the size of selem, which is non-ideal.

Fixes a10787e6d58c ("bpf: Enable task local storage for tracing programs")

---

Quoted 20 lines
 kernel/bpf/bpf_local_storage.c | 3 +++
 1 file changed, 3 insertions(+)
kernel/bpf/bpf_local_storage.c +3 -1
index 69bb2a7a71bc..cad17ca8552f 100644
@@ -393,6 +393,9 @@ int bpf_selem_unlink(struct bpf_local_storage_elem *selem)
        unsigned long flags;
        int err;
 
+       if (in_nmi())
+               return -EOPNOTSUPP;
+
        if (unlikely(!selem_linked_to_storage_lockless(selem)))
                /* selem has already been unlinked from sk */
                return 0;
--

2.52.0

author
#2 lore raw
Quoted 8 lines · Kumar Kartikeya Dwivedi
Quoted 5 lines · Amery Hung

Currently, local storage may deadlock when deferring freeing selem or local storage through kfree_rcu(), call_rcu() or call_rcu_tasks_trace() in NMI or reentrant. kfree_rcu() and call_rcu() have always been unsafe in NMI. call_rcu_tasks_trace() only became unsafe when it switched to

Quoted 5 lines

It has also always been unsafe in NMI, so the commit log needs an update. We should also note that the correct solution is irqs_disabled() to completely close the same context reentrancy, but it is too coarse-grained for this case, and we can revisit it once we convert to the _nolock() variant.

Thanks for the clarification on the call_rcu_tasks_trace(). I will adjust the commit msg and send v2.

Quoted 38 lines
Quoted 8 lines

call_srcu(). Since deleting selem in NMI is an unlikely use case, partially mitigate it by returning error when calling from bpf_xxx_storage_delete() helpers in NMI. The check is only added to bpf_selem_unlink(), which is used by helpers and syscalls. bpf_selem_unlink_nofail() is fine as it is called during map and owner tear down that never run in NMI or reentrant.

Quoted 7 lines

The long-term solution requires _nolock versions of call_rcu. Another possible solution is to defer the free through irq_work, but it would grow the size of selem, which is non-ideal.

Fixes a10787e6d58c ("bpf: Enable task local storage for tracing programs")

---

Quoted 20 lines
 kernel/bpf/bpf_local_storage.c | 3 +++
 1 file changed, 3 insertions(+)
kernel/bpf/bpf_local_storage.c +3 -1
index 69bb2a7a71bc..cad17ca8552f 100644
@@ -393,6 +393,9 @@ int bpf_selem_unlink(struct bpf_local_storage_elem *selem)
        unsigned long flags;
        int err;
 
+       if (in_nmi())
+               return -EOPNOTSUPP;
+
        if (unlikely(!selem_linked_to_storage_lockless(selem)))
                /* selem has already been unlinked from sk */
                return 0;
--

2.52.0

Reviewed-by on v1 1/1
#3 lore raw
In reply to Amery Hung
Quoted 4 lines · Amery Hung

Currently, local storage may deadlock when deferring freeing selem or local storage through kfree_rcu(), call_rcu() or call_rcu_tasks_trace() in NMI or reentrant.

The fix looks correct. Adding the in_nmi() check in bpf_selem_unlink() rather than in each individual helper is the right approach, as it centralizes the protection at the single entry point used by all bpf_xxx_storage_delete() helpers.

As Kumar noted, the commit log should mention that kfree_rcu() and call_rcu() have always been unsafe in NMI, not just call_rcu_tasks_trace().

One question: should the return value be -EOPNOTSUPP or would -EBUSY be more appropriate here? -EOPNOTSUPP typically means the operation is not supported at all, while -EBUSY might better convey that the operation cannot be performed in the current context but could succeed otherwise.

Otherwise the patch looks good to me.