Skip to contentSkip to conversation

bpf: Avoid unnecessary -EBUSY from htab_lock_bucket

PATCH Applied Song Liu 2 participants 2 messages Oct 12, 2023 – Oct 13, 2023 BPF

lore
Share & tools
lore

Song Liu

author reviewer
#0 lore raw

PATCH 1/1 bpf: Avoid unnecessary -EBUSY from htab_lock_bucket

htab_lock_bucket uses the following logic to avoid recursion:

1. preempt_disable();
2. check percpu counter htab->map_locked[hash] for recursion; 2.1. if map_lock[hash] is already taken, return -BUSY;
3. raw_spin_lock_irqsave();

However, if an IRQ hits between 2 and 3, BPF programs attached to the IRQ logic will not able to access the same hash of the hashtab and get -EBUSY. This -EBUSY is not really necessary. Fix it by disabling IRQ before checking map_locked:

1. preempt_disable();
2. local_irq_save();
3. check percpu counter htab->map_locked[hash] for recursion; 3.1. if map_lock[hash] is already taken, return -BUSY;
4. raw_spin_lock().

Similarly, use raw_spin_unlock() and local_irq_restore() in htab_unlock_bucket().

---
Resend to bpf tree to run CI with Ilya's s390 fixes.

---
 kernel/bpf/hashtab.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
kernel/bpf/hashtab.c +5 -2
index a8c7e1c5abfa..fd8d4b0addfc 100644
@@ -155,13 +155,15 @@ static inline int htab_lock_bucket(const struct bpf_htab *htab,
 	hash = hash & min_t(u32, HASHTAB_MAP_LOCK_MASK, htab->n_buckets - 1);
 
 	preempt_disable();
+	local_irq_save(flags);
 	if (unlikely(__this_cpu_inc_return(*(htab->map_locked[hash])) != 1)) {
 		__this_cpu_dec(*(htab->map_locked[hash]));
+		local_irq_restore(flags);
 		preempt_enable();
 		return -EBUSY;
 	}
 
-	raw_spin_lock_irqsave(&b->raw_lock, flags);
+	raw_spin_lock(&b->raw_lock);
 	*pflags = flags;
 
 	return 0;
@@ -172,8 +174,9 @@ static inline void htab_unlock_bucket(const struct bpf_htab *htab,
 				      unsigned long flags)
 {
 	hash = hash & min_t(u32, HASHTAB_MAP_LOCK_MASK, htab->n_buckets - 1);
-	raw_spin_unlock_irqrestore(&b->raw_lock, flags);
+	raw_spin_unlock(&b->raw_lock);
 	__this_cpu_dec(*(htab->map_locked[hash]));
+	local_irq_restore(flags);
 	preempt_enable();
 }
 
Signature · 1 line
2.34.1
#1 lore raw
This patch was applied to bpf/bpf.git (master) by Andrii Nakryiko <[email protected]>:

Hello:

This patch was applied to bpf/bpf.git (master)
by Andrii Nakryiko <[email protected]>:

Quoted 8 lines · On Wed, 11 Oct 2023 22:57:41 -0700 you wrote:

htab_lock_bucket uses the following logic to avoid recursion:

1. preempt_disable();
2. check percpu counter htab->map_locked[hash] for recursion; 2.1. if map_lock[hash] is already taken, return -BUSY;
3. raw_spin_lock_irqsave();

[...]

Here is the summary with links:
- [bpf] bpf: Avoid unnecessary -EBUSY from htab_lock_bucket

    https://git.kernel.org/bpf/bpf/c/a307f5a24924

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html