Skip to contentSkip to conversation

bpf: Zero-fill non-target CPU slots on BPF_F_CPU insert of new per-CPU element

PATCH In review Ömer Mete Kaya 3 participants 4 messages Sep 28 – Sep 29 BPF

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 bpf: Zero-fill non-target CPU slots on BPF_F_CPU insert of new per-CPU element

pcpu_copy_value() with BPF_F_CPU copies the value only to the target CPU's slot and returns immediately, leaving all other CPUs untouched. When inserting a *new* element, the backing per-CPU area may be reused from bpf_mem_cache or a prealloc freelist without being zero-initialized, so non-target CPUs retain stale contents of a previously-deleted element in the same map — a same-map information disclosure.

pcpu_init_value() is used to initialize newly allocated elements, while updates to existing elements call pcpu_copy_value() directly; non-target CPU slots intentionally retain their previous values there.

The onallcpus=false path in pcpu_init_value() already handles new element initialization correctly by explicitly zeroing non-current CPUs:

  if (cpu == current_cpu)
      copy_map_value(..., value);
  else
      zero_map_value(...);

Fix by adding a zero-fill pass in pcpu_init_value() after pcpu_copy_value() returns, covering only the BPF_F_CPU new-element
path: iterate all CPUs and zero-fill every slot except the target. This leaves pcpu_copy_value() unchanged so the update-existing-element semantics are not affected.

Fixes c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps")

---
Tested with a reproducer:
- Before the fix: CPU 1 retained 0xDEADBEEFCAFEBABE from the deleted element.
- After the fix: the non-target CPU read back zero.

 kernel/bpf/hashtab.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/bpf/hashtab.c +10 -0
index 8cc07e54daaa..dff4321f2828 100644
@@ -1099,6 +1099,16 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr,
 		}
 	} else {
 		pcpu_copy_value(htab, pptr, value, onallcpus, map_flags);
+		if (map_flags & BPF_F_CPU) {
+			int target = map_flags >> 32;
+			int cpu;
+
+			for_each_possible_cpu(cpu) {
+				if (cpu != target)
+					zero_map_value(&htab->map,
+						       per_cpu_ptr(pptr, cpu));
+			}
+		}
 	}
 }
 

--
2.55.0

#1 lore raw
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] bpf: unreachable dead code added in pcpu_init_value() --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] bpf: unreachable dead code added in pcpu_init_value()
--

commit aa9c1147874e7681c03a2eaf750ae6f2bf2a633a
Author: Ömer Mete Kaya <[email protected]>

bpf: Zero-fill non-target CPU slots on BPF_F_CPU insert of new per-CPU element

pcpu_copy_value() with BPF_F_CPU copies the value only to the target CPU's slot and returns immediately, leaving all other CPUs untouched. When inserting a *new* element, the backing per-CPU area may be reused from bpf_mem_cache or a prealloc freelist without being zero-initialized, so non-target CPUs retain stale contents of a previously-deleted element in the same map — a same-map information disclosure.

kernel/bpf/hashtab.c static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr, · line 1089

Quoted 9 lines
kernel/bpf/hashtab.c +1 -0
index 548eb1cc7c45c..f426d4c00bdc8 100644
@@ -1086,6 +1086,16 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr,
 		}
 	} else {
 		pcpu_copy_value(htab, pptr, value, onallcpus, map_flags);
+		if (map_flags & BPF_F_CPU) {

[Severity: Low]
Is this block reachable?

Since the preceding if statement in pcpu_init_value() evaluates "if (!onallcpus || (map_flags & BPF_F_CPU))", the else branch will only execute if "(map_flags & BPF_F_CPU)" evaluates to false.

This makes the newly added code here unreachable. Furthermore, the new element initialization issue mentioned in the commit message appears to already be handled correctly in the first branch, which already zeroes out non-target CPUs via zero_map_value().

Quoted 11 lines
kernel/bpf/hashtab.c +9 -0
+			int target = map_flags >> 32;
+			int cpu;
+
+			for_each_possible_cpu(cpu) {
+				if (cpu != target)
+					zero_map_value(&htab->map,
+						       per_cpu_ptr(pptr, cpu));
+			}
+		}
 	}
 }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#2 lore raw
In reply to Ömer Mete Kaya
Quoted 25 lines · Ömer Mete Kaya

pcpu_copy_value() with BPF_F_CPU copies the value only to the target CPU's slot and returns immediately, leaving all other CPUs untouched. When inserting a *new* element, the backing per-CPU area may be reused from bpf_mem_cache or a prealloc freelist without being zero-initialized, so non-target CPUs retain stale contents of a previously-deleted element in the same map — a same-map information disclosure.

pcpu_init_value() is used to initialize newly allocated elements, while updates to existing elements call pcpu_copy_value() directly; non-target CPU slots intentionally retain their previous values there.

The onallcpus=false path in pcpu_init_value() already handles new element initialization correctly by explicitly zeroing non-current CPUs:

  if (cpu == current_cpu)
      copy_map_value(..., value);
  else
      zero_map_value(...);

Fix by adding a zero-fill pass in pcpu_init_value() after pcpu_copy_value() returns, covering only the BPF_F_CPU new-element
path: iterate all CPUs and zero-fill every slot except the target. This leaves pcpu_copy_value() unchanged so the update-existing-element semantics are not affected.

Quoted 7 lines
Fixes c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps")

---
Tested with a reproducer:
- Before the fix: CPU 1 retained 0xDEADBEEFCAFEBABE from the deleted element.
- After the fix: the non-target CPU read back zero.

Could you add a selftest to prove the issue?

You said a reproducer. But, I don't know what it looks like.

Quoted 9 lines
 kernel/bpf/hashtab.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/bpf/hashtab.c +0 -0
index 8cc07e54daaa..dff4321f2828 100644

kernel/bpf/hashtab.c static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr, · line 1106

Quoted 8 lines
kernel/bpf/hashtab.c +5 -0
@@ -1099,6 +1099,16 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr,
 		}
 	} else {
 		pcpu_copy_value(htab, pptr, value, onallcpus, map_flags);
+		if (map_flags & BPF_F_CPU) {
+			int target = map_flags >> 32;
+			int cpu;
+
+			for_each_possible_cpu(cpu) {

But wait, the 'map_flags & BPF_F_CPU' has been check in the first 'if ()' condition in pcpu_init_value(). See the previous fix:
https://lore.kernel.org/bpf/[email protected]/

Thanks,
Leon

Quoted 10 lines
kernel/bpf/hashtab.c +5 -1
+				if (cpu != target)
+					zero_map_value(&htab->map,
+						       per_cpu_ptr(pptr, cpu));
+			}
+		}
 	}
 }
 
--

2.55.0

#3 lore raw
Quoted 51 lines · Leon Hwang
Quoted 25 lines · Ömer Mete Kaya

pcpu_copy_value() with BPF_F_CPU copies the value only to the target CPU's slot and returns immediately, leaving all other CPUs untouched. When inserting a *new* element, the backing per-CPU area may be reused from bpf_mem_cache or a prealloc freelist without being zero-initialized, so non-target CPUs retain stale contents of a previously-deleted element in the same map — a same-map information disclosure.

pcpu_init_value() is used to initialize newly allocated elements, while updates to existing elements call pcpu_copy_value() directly; non-target CPU slots intentionally retain their previous values there.

The onallcpus=false path in pcpu_init_value() already handles new element initialization correctly by explicitly zeroing non-current CPUs:

  if (cpu == current_cpu)
      copy_map_value(..., value);
  else
      zero_map_value(...);

Fix by adding a zero-fill pass in pcpu_init_value() after pcpu_copy_value() returns, covering only the BPF_F_CPU new-element
path: iterate all CPUs and zero-fill every slot except the target. This leaves pcpu_copy_value() unchanged so the update-existing-element semantics are not affected.

Quoted 7 lines
Fixes c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps")

---
Tested with a reproducer:
- Before the fix: CPU 1 retained 0xDEADBEEFCAFEBABE from the deleted element.
- After the fix: the non-target CPU read back zero.

Could you add a selftest to prove the issue?

You said a reproducer. But, I don't know what it looks like.

Quoted 13 lines
 kernel/bpf/hashtab.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/bpf/hashtab.c +1 -0
index 8cc07e54daaa..dff4321f2828 100644
@@ -1099,6 +1099,16 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr,
 		}
 	} else {
 		pcpu_copy_value(htab, pptr, value, onallcpus, map_flags);
+		if (map_flags & BPF_F_CPU) {
Quoted 8 lines

kernel/bpf/hashtab.c static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr, · line 1106

Quoted 4 lines
kernel/bpf/hashtab.c +4 -0
+			int target = map_flags >> 32;
+			int cpu;
+
+			for_each_possible_cpu(cpu) {

But wait, the 'map_flags & BPF_F_CPU' has been check in the first 'if ()' condition in pcpu_init_value(). See the previous fix:
https://lore.kernel.org/bpf/[email protected]/

Ah yes didn't see his fix, and it looks much cleaner.I think mine is no longer necessary.

Thanks,
Ömer