Skip to contentSkip to conversation

bpf: Use separate RCU callbacks for freeing selem

PATCH In review Kumar Kartikeya Dwivedi 2 participants 4 messages Mar 3, 2023

Subsystems (2)
lore
Share & tools
lore

Kumar Kartikeya Dwivedi

author maintainer
#0 lore raw

PATCH 1/1 bpf: Use separate RCU callbacks for freeing selem

Martin suggested that instead of using a byte in the hole (which he has a use for in his future patch) in bpf_local_storage_elem, we can dispatch a different call_rcu callback based on whether we need to free special fields in bpf_local_storage_elem data. The free path, described in commit 9db44fdd8105 ("bpf: Support kptrs in local storage maps"), only waits for call_rcu callbacks when there are special (kptrs, etc.) fields in the map value, hence it is necessary that we only access smap in this case.

Therefore, dispatch different RCU callbacks based on the BPF map has a valid btf_record, which dereference and use smap's btf_record only when it is valid.

---
 include/linux/bpf_local_storage.h |  6 ---
 kernel/bpf/bpf_local_storage.c    | 76 +++++++++++++++++++------------
 2 files changed, 46 insertions(+), 36 deletions(-)
include/linux/bpf_local_storage.h +0 -6
index 0fe92986412b..6d37a40cd90e 100644
@@ -74,12 +74,6 @@ struct bpf_local_storage_elem {
 	struct hlist_node snode;	/* Linked to bpf_local_storage */
 	struct bpf_local_storage __rcu *local_storage;
 	struct rcu_head rcu;
-	bool can_use_smap; /* Is it safe to access smap in bpf_selem_free_* RCU
-			    * callbacks? bpf_local_storage_map_free only
-			    * executes rcu_barrier when there are special
-			    * fields, this field remembers that to ensure we
-			    * don't access already freed smap in sdata.
-			    */
 	/* 8 bytes hole */
 	/* The data is stored in another cacheline to minimize
 	 * the number of cachelines access during a cache hit.
kernel/bpf/bpf_local_storage.c +46 -30

122 lines not on this page · raw message

--
2.39.2

#1 lore raw
Quoted 9 lines · Kumar Kartikeya Dwivedi

Martin suggested that instead of using a byte in the hole (which he has a use for in his future patch) in bpf_local_storage_elem, we can dispatch a different call_rcu callback based on whether we need to free special fields in bpf_local_storage_elem data. The free path, described in commit 9db44fdd8105 ("bpf: Support kptrs in local storage maps"), only waits for call_rcu callbacks when there are special (kptrs, etc.) fields in the map value, hence it is necessary that we only access smap in this case.

Quoted 5 lines

Therefore, dispatch different RCU callbacks based on the BPF map has a valid btf_record, which dereference and use smap's btf_record only when it is valid.

Thanks for your patch. I have already made a similar change in my local branch which has some differences like refactored it a little for my work. The set is almost ready. Do you mind if I include your patch in my set and keep your SOB?

author maintainer
#2 lore raw
Quoted 15 lines · Martin KaFai Lau
Quoted 13 lines · Kumar Kartikeya Dwivedi

Martin suggested that instead of using a byte in the hole (which he has a use for in his future patch) in bpf_local_storage_elem, we can dispatch a different call_rcu callback based on whether we need to free special fields in bpf_local_storage_elem data. The free path, described in commit 9db44fdd8105 ("bpf: Support kptrs in local storage maps"), only waits for call_rcu callbacks when there are special (kptrs, etc.) fields in the map value, hence it is necessary that we only access smap in this case.

Therefore, dispatch different RCU callbacks based on the BPF map has a valid btf_record, which dereference and use smap's btf_record only when it is valid.

Quoted 6 lines
Quoted 1 line

Thanks for your patch. I have already made a similar change in my local branch which has some differences like refactored it a little for my work. The set is almost ready. Do you mind if I include your patch in my set and keep your SOB?

No problem, please do.

#3 lore raw
Quoted 16 lines · Kumar Kartikeya Dwivedi
Quoted 15 lines · Martin KaFai Lau
Quoted 13 lines · Kumar Kartikeya Dwivedi

Martin suggested that instead of using a byte in the hole (which he has a use for in his future patch) in bpf_local_storage_elem, we can dispatch a different call_rcu callback based on whether we need to free special fields in bpf_local_storage_elem data. The free path, described in commit 9db44fdd8105 ("bpf: Support kptrs in local storage maps"), only waits for call_rcu callbacks when there are special (kptrs, etc.) fields in the map value, hence it is necessary that we only access smap in this case.

Therefore, dispatch different RCU callbacks based on the BPF map has a valid btf_record, which dereference and use smap's btf_record only when it is valid.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

Thanks for your patch. I have already made a similar change in my local branch which has some differences like refactored it a little for my work. The set is almost ready. Do you mind if I include your patch in my set and keep your SOB?

No problem, please do.

Please ignore my previous message. I will make some adjustments on my set.

Applied with some changes on de-referencing SDATA(selem)->smap to address these warnings:

# this is addressed by rcu_dereference_protected(..., true)
../kernel/bpf/bpf_local_storage.c:117:41: warning: dereference of noderef expression
# this is addressed by directly using the earlier dereferenced smap pointer
../kernel/bpf/bpf_local_storage.c:200:27: warning: dereference of noderef expression