bpf: fix BPF_F_CPU validation for sparse CPU IDs
PATCH 1/1 bpf: fix BPF_F_CPU validation for sparse CPU IDs
BPF_F_CPU stores the target CPU ID in the upper 32 bits of the map operation flags. bpf_map_check_op_flags() currently compares that ID with num_possible_cpus(), which is the number of possible CPUs rather than a bound on CPU IDs.
On an arm64 QEMU guest with a CPU device-tree hole, the possible CPU mask was 0,2-3. A userspace program using raw bpf() syscalls creates a BPF_MAP_TYPE_PERCPU_ARRAY and performs update and lookup operations for each CPU by setting BPF_F_CPU and the CPU ID in the flags.
With the old check, CPU 1 is incorrectly accepted while valid CPU 3 is rejected with -ERANGE. The CPU 1 update then reaches the per-CPU map access path and triggers:
Unable to handle kernel paging request at virtual address ...
pc : __pi_memcpy_generic+0x5c/0x22c
lr : bpf_percpu_array_update+0x2dc/0x2e8
Call trace:
__pi_memcpy_generic
bpf_map_update_value
map_update_elem
__sys_bpfCheck the CPU ID against nr_cpu_ids and cpu_possible() instead. This rejects CPU IDs outside the valid range and CPUs absent from the possible mask, while allowing valid sparse CPU IDs.
--- include/linux/bpf.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
include/linux/bpf.h +2 -1
@@ -4202,7 +4202,8 @@ static inline int bpf_map_check_op_flags(struct bpf_map *map, u64 flags, u64 allreturn -EINVAL ; cpu = flags >>32 ; -if ((flags &BPF_F_CPU ) && cpu >=num_possible_cpus ()) +if ((flags &BPF_F_CPU ) && + (cpu >= nr_cpu_ids || !cpu_possible (cpu)))return -ERANGE ; }
Signature · 1 line
2.54.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Bulk updates of per-CPU maps (without BPF_F_CPU or BPF_F_ALL_CPUS flags) read out-of-bounds from the userspace-provided value buffer on systems with sparse CPU possible masks. --
--- Patch [1]: [PATCH bpf] bpf: fix BPF_F_CPU validation for sparse CPU IDs --- commit 821903ca90d43b9f0e29ab7849f998f149e58245
Author: Hui Su <[email protected]>
bpf: fix BPF_F_CPU validation for sparse CPU IDs
BPF_F_CPU stores the target CPU ID in the upper 32 bits of the map operation flags. bpf_map_check_op_flags() currently compares that ID with num_possible_cpus(), which is the number of possible CPUs rather than a bound on CPU IDs.
include/linux/bpf.h static inline int bpf_map_check_op_flags(struct bpf_map *map, u64 flags, u64 all · line 4207
Quoted 13 lines
include/linux/bpf.h +2
-1@@ -4201,7 +4201,8 @@ static inline int bpf_map_check_op_flags(struct bpf_map *map, u64 flags, u64 allreturn -EINVAL ; cpu = flags >>32 ; -if ((flags &BPF_F_CPU ) && cpu >=num_possible_cpus ()) +if ((flags &BPF_F_CPU ) && + (cpu >= nr_cpu_ids || !cpu_possible (cpu)))return -ERANGE ; }
[Severity: High]
This is a pre-existing issue, but does the bulk update path (without BPF_F_CPU or BPF_F_ALL_CPUS) have an out-of-bounds read on sparse CPU masks?
When a per-CPU map is updated from userspace, the user buffer is copied into a kernel buffer allocated densely based on num_possible_cpus():
kernel/bpf/syscall.c:bpf_map_value_size() {
...
return round_up(map->value_size, 8) * num_possible_cpus();
...
}However, during bulk update, the source pointer is calculated sparsely using the cpu index:
kernel/bpf/arraymap.c:bpf_percpu_array_update() {
...
for_each_possible_cpu(cpu) {
...
val = (map_flags & BPF_F_ALL_CPUS) ? value : value + size * cpu;
copy_map_value(map, ptr, val);
...
}If the possible CPUs are sparse (e.g., 0, 2, 3), num_possible_cpus() is 3, making the allocated buffer size * 3 bytes long. But in the loop, cpu can reach 3, resulting in value + size * 3, which points past the end of the allocated buffer.
Can this lead to an out-of-bounds read that copies leaked kernel heap data into the BPF map? This pattern also appears in hashtab.c:pcpu_copy_value() and local_storage.c:bpf_percpu_cgroup_storage_update().
Note that lookup operations (like bpf_percpu_array_copy) seem to use dense indexing (off += size), which suggests a mismatch between how values are read and written in bulk.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Hi Hui,
Quoted 6 lines · Hui Su
BPF_F_CPUstores the target CPU ID in the upper 32 bits of the map operation flags.bpf_map_check_op_flags()currently compares that ID withnum_possible_cpus(), which is the number of possible CPUs rather than a bound on CPU IDs.On an arm64 QEMU guest with a CPU device-tree hole, the possible CPU
TIL about the possible CPU hole.
Quoted 32 lines
mask was 0,2-3. A userspace program using raw
bpf()syscalls creates aBPF_MAP_TYPE_PERCPU_ARRAYand performs update and lookup operations for each CPU by settingBPF_F_CPUand the CPU ID in the flags.With the old check, CPU 1 is incorrectly accepted while valid CPU 3 is rejected with
-ERANGE. The CPU 1 update then reaches the per-CPU map access path and triggers:Unable to handle kernel paging request at virtual address ... pc : __pi_memcpy_generic+0x5c/0x22c lr : bpf_percpu_array_update+0x2dc/0x2e8 Call trace: __pi_memcpy_generic bpf_map_update_value map_update_elem __sys_bpfCheck the CPU ID against
nr_cpu_idsandcpu_possible()instead. This rejects CPU IDs outside the valid range and CPUs absent from the possible mask, while allowing valid sparse CPU IDs.Fixes 2b421662c788 ("bpf: Introduce BPF_F_CPU and BPF_F_ALL_CPUS flags")Signed-off-by Hui Su <[email protected]>--- include/linux/bpf.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)include/linux/bpf.h +0
-0
include/linux/bpf.h static inline int bpf_map_check_op_flags(struct bpf_map *map, u64 flags, u64 all · line 4206
Quoted 6 lines
include/linux/bpf.h +2
-1@@ -4202,7 +4202,8 @@ static inline int bpf_map_check_op_flags(struct bpf_map *map, u64 flags, u64 allreturn -EINVAL ; cpu = flags >>32 ; -if ((flags &BPF_F_CPU ) && cpu >=num_possible_cpus ()) +if ((flags &BPF_F_CPU ) && + (cpu >= nr_cpu_ids || !cpu_possible (cpu)))
NIT: no need to put the cpu check on a new line, as 100 chars per line are allowed.
Other than the NIT,
Quoted 3 lines
include/linux/bpf.h +0
-0return -ERANGE ; }