bpf: Zero-fill non-target CPU slots on BPF_F_CPU insert of new per-CPU element
PATCH 1/1 bpf: Zero-fill non-target CPU slots on BPF_F_CPU insert of new per-CPU element
pcpu_copy_value() with BPF_F_CPU copies the value only to the target CPU's slot and returns immediately, leaving all other CPUs untouched. When inserting a *new* element, the backing per-CPU area may be reused from bpf_mem_cache or a prealloc freelist without being zero-initialized, so non-target CPUs retain stale contents of a previously-deleted element in the same map — a same-map information disclosure.
pcpu_init_value() is used to initialize newly allocated elements, while updates to existing elements call pcpu_copy_value() directly; non-target CPU slots intentionally retain their previous values there.
The onallcpus=false path in pcpu_init_value() already handles new element initialization correctly by explicitly zeroing non-current CPUs:
if (cpu == current_cpu)
copy_map_value(..., value);
else
zero_map_value(...);Fix by adding a zero-fill pass in pcpu_init_value() after pcpu_copy_value() returns, covering only the BPF_F_CPU new-element
path: iterate all CPUs and zero-fill every slot except the target. This leaves pcpu_copy_value() unchanged so the update-existing-element semantics are not affected.
---
Tested with a reproducer:
- Before the fix: CPU 1 retained 0xDEADBEEFCAFEBABE from the deleted element.
- After the fix: the non-target CPU read back zero.
kernel/bpf/hashtab.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
kernel/bpf/hashtab.c +10 -0
@@ -1099,6 +1099,16 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr, } }else {pcpu_copy_value (htab, pptr, value, onallcpus, map_flags); +if (map_flags &BPF_F_CPU ) { +int target = map_flags >>32 ; +int cpu; + +for_each_possible_cpu (cpu) { +if (cpu != target) +zero_map_value (&htab->map, +per_cpu_ptr (pptr, cpu)); + } + } } }
--
2.55.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] bpf: unreachable dead code added in pcpu_init_value() --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] bpf: unreachable dead code added in pcpu_init_value()
--
commit aa9c1147874e7681c03a2eaf750ae6f2bf2a633a
Author: Ömer Mete Kaya <[email protected]>
bpf: Zero-fill non-target CPU slots on BPF_F_CPU insert of new per-CPU element
pcpu_copy_value() with BPF_F_CPU copies the value only to the target CPU's slot and returns immediately, leaving all other CPUs untouched. When inserting a *new* element, the backing per-CPU area may be reused from bpf_mem_cache or a prealloc freelist without being zero-initialized, so non-target CPUs retain stale contents of a previously-deleted element in the same map — a same-map information disclosure.
kernel/bpf/hashtab.c static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr, · line 1089
Quoted 9 lines
kernel/bpf/hashtab.c +1
-0@@ -1086,6 +1086,16 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr, } }else {pcpu_copy_value (htab, pptr, value, onallcpus, map_flags); +if (map_flags &BPF_F_CPU ) {
[Severity: Low]
Is this block reachable?
Since the preceding if statement in pcpu_init_value() evaluates "if (!onallcpus || (map_flags & BPF_F_CPU))", the else branch will only execute if "(map_flags & BPF_F_CPU)" evaluates to false.
This makes the newly added code here unreachable. Furthermore, the new element initialization issue mentioned in the commit message appears to already be handled correctly in the first branch, which already zeroes out non-target CPUs via zero_map_value().
Quoted 11 lines
kernel/bpf/hashtab.c +9
-0+int target = map_flags >>32 ; +int cpu; + +for_each_possible_cpu (cpu) { +if (cpu != target) +zero_map_value (&htab->map, +per_cpu_ptr (pptr, cpu)); + } + } } }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Quoted 25 lines · Ömer Mete Kaya
pcpu_copy_value()withBPF_F_CPUcopies the value only to the target CPU's slot and returns immediately, leaving all other CPUs untouched. When inserting a *new* element, the backing per-CPU area may be reused frombpf_mem_cacheor a prealloc freelist without being zero-initialized, so non-target CPUs retain stale contents of a previously-deleted element in the same map — a same-map information disclosure.
pcpu_init_value()is used to initialize newly allocated elements, while updates to existing elements callpcpu_copy_value()directly; non-target CPU slots intentionally retain their previous values there.The onallcpus=false path in
pcpu_init_value()already handles new element initialization correctly by explicitly zeroing non-current CPUs:if (cpu == current_cpu) copy_map_value(..., value); else zero_map_value(...);Fix by adding a zero-fill pass in
pcpu_init_value()afterpcpu_copy_value()returns, covering only theBPF_F_CPUnew-element
path: iterate all CPUs and zero-fill every slot except the target. This leavespcpu_copy_value()unchanged so the update-existing-element semantics are not affected.
Quoted 7 lines
Fixes c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps")Signed-off-by Ömer Mete Kaya <[email protected]>---
Tested with a reproducer:
- Before the fix: CPU 1 retained0xDEADBEEFCAFEBABEfrom the deleted element.
- After the fix: the non-target CPU read back zero.
Could you add a selftest to prove the issue?
You said a reproducer. But, I don't know what it looks like.
Quoted 9 lines
kernel/bpf/hashtab.c | 10 ++++++++++ 1 file changed, 10 insertions(+)kernel/bpf/hashtab.c +0
-0
kernel/bpf/hashtab.c static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr, · line 1106
Quoted 8 lines
kernel/bpf/hashtab.c +5
-0@@ -1099,6 +1099,16 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr, } }else {pcpu_copy_value (htab, pptr, value, onallcpus, map_flags); +if (map_flags &BPF_F_CPU ) { +int target = map_flags >>32 ; +int cpu; + +for_each_possible_cpu (cpu) {
But wait, the 'map_flags & BPF_F_CPU' has been check in the first 'if ()' condition in pcpu_init_value(). See the previous fix:
https://lore.kernel.org/bpf/[email protected]/
Thanks,
Leon
Quoted 10 lines
kernel/bpf/hashtab.c +5
-1+if (cpu != target) +zero_map_value (&htab->map, +per_cpu_ptr (pptr, cpu)); + } + } } } --2.55.0
Quoted 51 lines · Leon Hwang
Quoted 25 lines · Ömer Mete Kaya
pcpu_copy_value()withBPF_F_CPUcopies the value only to the target CPU's slot and returns immediately, leaving all other CPUs untouched. When inserting a *new* element, the backing per-CPU area may be reused frombpf_mem_cacheor a prealloc freelist without being zero-initialized, so non-target CPUs retain stale contents of a previously-deleted element in the same map — a same-map information disclosure.
pcpu_init_value()is used to initialize newly allocated elements, while updates to existing elements callpcpu_copy_value()directly; non-target CPU slots intentionally retain their previous values there.The onallcpus=false path in
pcpu_init_value()already handles new element initialization correctly by explicitly zeroing non-current CPUs:if (cpu == current_cpu) copy_map_value(..., value); else zero_map_value(...);Fix by adding a zero-fill pass in
pcpu_init_value()afterpcpu_copy_value()returns, covering only theBPF_F_CPUnew-element
path: iterate all CPUs and zero-fill every slot except the target. This leavespcpu_copy_value()unchanged so the update-existing-element semantics are not affected.Quoted 7 lines
Fixes c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps")Signed-off-by Ömer Mete Kaya <[email protected]>---
Tested with a reproducer:
- Before the fix: CPU 1 retained0xDEADBEEFCAFEBABEfrom the deleted element.
- After the fix: the non-target CPU read back zero.Could you add a selftest to prove the issue?
You said a reproducer. But, I don't know what it looks like.
Quoted 13 lines
kernel/bpf/hashtab.c | 10 ++++++++++ 1 file changed, 10 insertions(+)kernel/bpf/hashtab.c +1
-0@@ -1099,6 +1099,16 @@ static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr, } }else {pcpu_copy_value (htab, pptr, value, onallcpus, map_flags); +if (map_flags &BPF_F_CPU ) {
Quoted 8 lines
kernel/bpf/hashtab.c static void pcpu_init_value(struct bpf_htab *htab, void __percpu *pptr, · line 1106
Quoted 4 lines
kernel/bpf/hashtab.c +4
-0+int target = map_flags >>32 ; +int cpu; + +for_each_possible_cpu (cpu) {But wait, the '
map_flags&BPF_F_CPU' has been check in the first 'if ()' condition inpcpu_init_value(). See the previous fix:
https://lore.kernel.org/bpf/[email protected]/
Ah yes didn't see his fix, and it looks much cleaner.I think mine is no longer necessary.
Thanks,
Ömer