Skip to contentSkip to conversation

bpf: fix BPF_F_CPU validation for sparse CPU IDs

PATCH Superseded Hui Su 3 participants 3 messages Aug 13

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Hui Su

author
#0 lore raw

PATCH 1/1 bpf: fix BPF_F_CPU validation for sparse CPU IDs

BPF_F_CPU stores the target CPU ID in the upper 32 bits of the map operation flags. bpf_map_check_op_flags() currently compares that ID with num_possible_cpus(), which is the number of possible CPUs rather than a bound on CPU IDs.

On an arm64 QEMU guest with a CPU device-tree hole, the possible CPU mask was 0,2-3. A userspace program using raw bpf() syscalls creates a BPF_MAP_TYPE_PERCPU_ARRAY and performs update and lookup operations for each CPU by setting BPF_F_CPU and the CPU ID in the flags.

With the old check, CPU 1 is incorrectly accepted while valid CPU 3 is rejected with -ERANGE. The CPU 1 update then reaches the per-CPU map access path and triggers:

  Unable to handle kernel paging request at virtual address ...
  pc : __pi_memcpy_generic+0x5c/0x22c
  lr : bpf_percpu_array_update+0x2dc/0x2e8
  Call trace:
    __pi_memcpy_generic
    bpf_map_update_value
    map_update_elem
    __sys_bpf

Check the CPU ID against nr_cpu_ids and cpu_possible() instead. This rejects CPU IDs outside the valid range and CPUs absent from the possible mask, while allowing valid sparse CPU IDs.

Fixes 2b421662c788 ("bpf: Introduce BPF_F_CPU and BPF_F_ALL_CPUS flags")
---
 include/linux/bpf.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
include/linux/bpf.h +2 -1
index 7719f6528445..282f7022d984 100644
@@ -4202,7 +4202,8 @@ static inline int bpf_map_check_op_flags(struct bpf_map *map, u64 flags, u64 all
 			return -EINVAL;
 
 		cpu = flags >> 32;
-		if ((flags & BPF_F_CPU) && cpu >= num_possible_cpus())
+		if ((flags & BPF_F_CPU) &&
+		    (cpu >= nr_cpu_ids || !cpu_possible(cpu)))
 			return -ERANGE;
 	}
 
Signature · 1 line
2.54.0
#1 lore raw
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] Bulk updates of per-CPU maps (without BPF_F_CPU or BPF_F_ALL_CPUS flags) read out-of-bounds from the userspace-provided value buffer on systems with sparse CPU possible masks. --

--- Patch [1]: [PATCH bpf] bpf: fix BPF_F_CPU validation for sparse CPU IDs --- commit 821903ca90d43b9f0e29ab7849f998f149e58245
Author: Hui Su <[email protected]>

bpf: fix BPF_F_CPU validation for sparse CPU IDs

BPF_F_CPU stores the target CPU ID in the upper 32 bits of the map operation flags. bpf_map_check_op_flags() currently compares that ID with num_possible_cpus(), which is the number of possible CPUs rather than a bound on CPU IDs.

include/linux/bpf.h static inline int bpf_map_check_op_flags(struct bpf_map *map, u64 flags, u64 all · line 4207

Quoted 13 lines
include/linux/bpf.h +2 -1
index b4a10c9878cf8..3d9a632d7d33d 100644
@@ -4201,7 +4201,8 @@ static inline int bpf_map_check_op_flags(struct bpf_map *map, u64 flags, u64 all
 			return -EINVAL;
 
 		cpu = flags >> 32;
-		if ((flags & BPF_F_CPU) && cpu >= num_possible_cpus())
+		if ((flags & BPF_F_CPU) &&
+		    (cpu >= nr_cpu_ids || !cpu_possible(cpu)))
 			return -ERANGE;
 	}

[Severity: High]
This is a pre-existing issue, but does the bulk update path (without BPF_F_CPU or BPF_F_ALL_CPUS) have an out-of-bounds read on sparse CPU masks?

When a per-CPU map is updated from userspace, the user buffer is copied into a kernel buffer allocated densely based on num_possible_cpus():

kernel/bpf/syscall.c:bpf_map_value_size() {
    ...
    return round_up(map->value_size, 8) * num_possible_cpus();
    ...
}

However, during bulk update, the source pointer is calculated sparsely using the cpu index:

kernel/bpf/arraymap.c:bpf_percpu_array_update() {
    ...
    for_each_possible_cpu(cpu) {
        ...
        val = (map_flags & BPF_F_ALL_CPUS) ? value : value + size * cpu;
        copy_map_value(map, ptr, val);
    ...
}

If the possible CPUs are sparse (e.g., 0, 2, 3), num_possible_cpus() is 3, making the allocated buffer size * 3 bytes long. But in the loop, cpu can reach 3, resulting in value + size * 3, which points past the end of the allocated buffer.

Can this lead to an out-of-bounds read that copies leaked kernel heap data into the BPF map? This pattern also appears in hashtab.c:pcpu_copy_value() and local_storage.c:bpf_percpu_cgroup_storage_update().

Note that lookup operations (like bpf_percpu_array_copy) seem to use dense indexing (off += size), which suggests a mismatch between how values are read and written in bulk.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Acked-by on v1 1/1
#2 lore raw
In reply to Hui Su

Hi Hui,

Quoted 6 lines · Hui Su

BPF_F_CPU stores the target CPU ID in the upper 32 bits of the map operation flags. bpf_map_check_op_flags() currently compares that ID with num_possible_cpus(), which is the number of possible CPUs rather than a bound on CPU IDs.

On an arm64 QEMU guest with a CPU device-tree hole, the possible CPU

TIL about the possible CPU hole.

Quoted 32 lines

mask was 0,2-3. A userspace program using raw bpf() syscalls creates a BPF_MAP_TYPE_PERCPU_ARRAY and performs update and lookup operations for each CPU by setting BPF_F_CPU and the CPU ID in the flags.

With the old check, CPU 1 is incorrectly accepted while valid CPU 3 is rejected with -ERANGE. The CPU 1 update then reaches the per-CPU map access path and triggers:

  Unable to handle kernel paging request at virtual address ...
  pc : __pi_memcpy_generic+0x5c/0x22c
  lr : bpf_percpu_array_update+0x2dc/0x2e8
  Call trace:
    __pi_memcpy_generic
    bpf_map_update_value
    map_update_elem
    __sys_bpf

Check the CPU ID against nr_cpu_ids and cpu_possible() instead. This rejects CPU IDs outside the valid range and CPUs absent from the possible mask, while allowing valid sparse CPU IDs.

Fixes 2b421662c788 ("bpf: Introduce BPF_F_CPU and BPF_F_ALL_CPUS flags")
---
 include/linux/bpf.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
include/linux/bpf.h +0 -0
index 7719f6528445..282f7022d984 100644

include/linux/bpf.h static inline int bpf_map_check_op_flags(struct bpf_map *map, u64 flags, u64 all · line 4206

Quoted 6 lines
include/linux/bpf.h +2 -1
@@ -4202,7 +4202,8 @@ static inline int bpf_map_check_op_flags(struct bpf_map *map, u64 flags, u64 all
 			return -EINVAL;
 
 		cpu = flags >> 32;
-		if ((flags & BPF_F_CPU) && cpu >= num_possible_cpus())
+		if ((flags & BPF_F_CPU) &&
+		    (cpu >= nr_cpu_ids || !cpu_possible(cpu)))

NIT: no need to put the cpu check on a new line, as 100 chars per line are allowed.

Other than the NIT,

Quoted 3 lines
include/linux/bpf.h +0 -0
 			return -ERANGE;
 	}