bpf: Fix incorrect handling of user flags by percpu map updates
Subsystems (4)
v5 cover bpf: Fix incorrect handling of user flags by percpu map updates
For BPF_MAP_TYPE_PERCPU_ARRAY map, bpf_percpu_array_update() is not considering the possibility of a combination of (BPF_NOEXIST, BPF_EXIST) flags with (BPF_F_CPU, BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST, BPF_EXIST) flags to lose their effect in some cases.
For BPF_MAP_TYPE_PERCPU_HASH and BPF_MAP_TYPE_LRU_PERCPU_HASH maps, htab_map_check_update_flags() and check_flags() are not considering the possibility of a combination of (BPF_NOEXIST, BPF_EXIST) flags with (BPF_F_CPU, BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST, BPF_EXIST) flags to lose their effect in some cases.
For example, when using (BPF_F_CPU | BPF_EXIST) flag combination with bpf_map_update_elem() on a BPF_MAP_TYPE_PERCPU_HASH map, the BPF_EXIST flag does not prevent new insertions as expected.
This series fixes the bug by adding proper flag validations and checks and adds regression tests.
V5 changes:
- Simplify flag validations by introducing new macros in bpf.h
- Add helpers to verify the values after updates in the new tests
- Improve the style and structure of the new tests
V4 changes: https://lore.kernel.org/bpf/[email protected]/T/
- Edit selftest to use proper value_sz in the new cgroup map test
- Edit selftest to pin the current pid to the current cpu for LRU map
- Edit commit messages to include user-visible changes
V3 changes: https://lore.kernel.org/bpf/[email protected]/T/
- Split the fix into two separate patches for array map and hash maps
- Extend the selftest to cover all percpu map types
- Extend the selftest to cover all applicable flag combinations
- Extend the selftest to cover the unnecessary delete issue of LRU map
V2 changes: https://lore.kernel.org/bpf/[email protected]/T/
- Fix a BPF_EXIST flag check in __htab_lru_percpu_map_update_elem()
- Add additional test for BPF_MAP_TYPE_LRU_PERCPU_HASH map
- Add check for BPF_EXIST, BPF_NOEXIST combination in percpu array map
V1: https://lore.kernel.org/bpf/[email protected]/T/
Masoud Aghasi (3):
bpf: Fix incorrect handling of user flags in bpf_percpu_array_update
bpf: Fix incorrect handling of user flags by percpu hash map updates selftests/bpf: add tests for percpu map flags combination
include/linux/bpf.h | 3 + kernel/bpf/arraymap.c | 6 +- kernel/bpf/hashtab.c | 10 +- .../selftests/bpf/prog_tests/percpu_alloc.c | 249 ++++++++++++++++++ 4 files changed, 261 insertions(+), 7 deletions(-)
Signature · 1 line
2.47.3
PATCH v5 1/3 bpf: Fix incorrect handling of user flags in bpf_percpu_array_update
For BPF_MAP_TYPE_PERCPU_ARRAY map, bpf_percpu_array_update() is not considering the possibility of a combination of (BPF_NOEXIST, BPF_EXIST) flags with (BPF_F_CPU, BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST, BPF_EXIST) flags to lose their effect in some cases.
For example, using the (BPF_F_ALL_CPUS | BPF_EXIST) flag combination with bpf_map_update_elem() results in an incorrect EINVAL error response, even though the flag combination is valid.
This patch fixes the bug by adding proper flag validations and checks. Before this patch, bpf_percpu_array_update() rejected BPF_F_ALL_CPUS | BPF_EXIST and BPF_F_ALL_CPUS | BPF_NOEXIST with -EINVAL. Also the BPF_F_CPU | BPF_NOEXIST were accepted. After the patch BPF_F_ALL_CPUS | BPF_EXIST is accepted and BPF_F_ALL_CPUS | BPF_NOEXIST and BPF_F_CPU | BPF_NOEXIST result in -EEXIST.
--- include/linux/bpf.h | 3 +++ kernel/bpf/arraymap.c | 6 +++--- 2 files changed, 6 insertions(+), 3 deletions(-)
include/linux/bpf.h +3 -0
@@ -4322,6 +4322,9 @@ static inline bool bpf_map_is_percpu_map(enum bpf_map_type map_type) } } +#define BPF_EXIST_FLAGS (BPF_EXIST | BPF_NOEXIST) +#define BPF_CPU_FLAGS (BPF_F_CPU | BPF_F_ALL_CPUS) +static inline int bpf_map_check_op_flags (struct bpf_map *map,u64 flags,u64 allowed_flags) {u32 cpu;
kernel/bpf/arraymap.c +3 -3
@@ -438,15 +438,15 @@ int bpf_percpu_array_update(struct bpf_map *map, void *key, void *value,u32 size;int cpu, off =0 ; -if (unlikely ((map_flags &BPF_F_LOCK ) || (u32 )map_flags >BPF_F_ALL_CPUS )) -/* unknown flags */ +if (unlikely (((map_flags &BPF_EXIST_FLAGS ) ==BPF_EXIST_FLAGS ) || + ((u32 )map_flags & ~(BPF_EXIST_FLAGS |BPF_CPU_FLAGS ))))return -EINVAL ;if (unlikely (index >= array->map.max_entries))/* all elements were pre-allocated, cannot insert a new one */ return -E2BIG ; -if (unlikely (map_flags ==BPF_NOEXIST )) +if (unlikely (map_flags &BPF_NOEXIST ))/* all elements already exist */ return -EEXIST ;
Signature · 1 line
2.47.3
[email protected]
AI reviewRe: [PATCH bpf v5 1/3] bpf: Fix incorrect handling of user flags in bpf_percpu_array_update
TL;DR: Commit message only: the patch also makes BPF_EXIST | BPF_NOEXIST return -EINVAL in bpf_percpu_array_update(), which was previously accepted, but the message doesn't mention it.
Quoted 1 line
bpf: Fix incorrect handling of user flags in
bpf_percpu_array_update
TL;DR: Commit message only: the patch also makes BPF_EXIST | BPF_NOEXIST return -EINVAL in bpf_percpu_array_update(), which was previously accepted, but the message doesn't mention it.
Quoted 20 lines
bpf: Fix incorrect handling of user flags in
bpf_percpu_array_updateFor
BPF_MAP_TYPE_PERCPU_ARRAYmap,bpf_percpu_array_update()is not considering the possibility of a combination of (BPF_NOEXIST,BPF_EXIST) flags with (BPF_F_CPU,BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST,BPF_EXIST) flags to lose their effect in some cases.For example, using the (
BPF_F_ALL_CPUS|BPF_EXIST) flag combination withbpf_map_update_elem()results in an incorrect EINVAL error response, even though the flag combination is valid.This patch fixes the bug by adding proper flag validations and checks. Before this patch,
bpf_percpu_array_update()rejectedBPF_F_ALL_CPUS|BPF_EXISTandBPF_F_ALL_CPUS|BPF_NOEXISTwith-EINVAL. Also theBPF_F_CPU|BPF_NOEXISTwere accepted. After the patchBPF_F_ALL_CPUS|BPF_EXISTis accepted andBPF_F_ALL_CPUS|BPF_NOEXISTandBPF_F_CPU|BPF_NOEXISTresult in-EEXIST.
Quoted 2 lines
Fixes 8eb76cb03f0f ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_array maps")Signed-off-by Masoud Aghasi <[email protected]>
The before/after paragraph lists BPF_F_ALL_CPUS | BPF_EXIST, BPF_F_ALL_CPUS | BPF_NOEXIST and BPF_F_CPU | BPF_NOEXIST. Is there a behaviour change here that isn't listed?
The new check in bpf_percpu_array_update() is:
if (unlikely(((map_flags & BPF_EXIST_FLAGS) == BPF_EXIST_FLAGS) ||
((u32)map_flags & ~(BPF_EXIST_FLAGS | BPF_CPU_FLAGS))))
return -EINVAL;so BPF_NOEXIST | BPF_EXIST, with or without BPF_F_CPU, now returns -EINVAL.
Before this patch, the old (u32)map_flags > BPF_F_ALL_CPUS test let both of those values through. map_flags == BPF_NOEXIST was false for them, so the update went ahead and returned 0:
map_update_elem()
bpf_map_check_op_flags(map, attr->flags, ~0)
bpf_map_update_value()
bpf_percpu_array_update()
The new behaviour matches what the code did before 8eb76cb03f0f, which rejected map_flags > BPF_EXIST.
The companion hash map patch in this series (e5859394034d, "bpf: Fix incorrect handling of user flags by percpu hash map updates") documents the same change:
This patch also starts returning -EINVAL for (BPF_EXIST | BPF_NOEXIST), with or without BPF_F_CPU, which was previously accepted.
Should a matching sentence be added to this commit message?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/37439873908
Re: [PATCH bpf v5 1/3] bpf: Fix incorrect handling of user flags in bpf_percpu_array_update
In reply to Masoud AghasiQuoted 18 lines · Masoud Aghasi
For
BPF_MAP_TYPE_PERCPU_ARRAYmap,bpf_percpu_array_update()is not considering the possibility of a combination of (BPF_NOEXIST,BPF_EXIST) flags with (BPF_F_CPU,BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST,BPF_EXIST) flags to lose their effect in some cases.For example, using the (
BPF_F_ALL_CPUS|BPF_EXIST) flag combination withbpf_map_update_elem()results in an incorrect EINVAL error response, even though the flag combination is valid.This patch fixes the bug by adding proper flag validations and checks. Before this patch,
bpf_percpu_array_update()rejectedBPF_F_ALL_CPUS|BPF_EXISTandBPF_F_ALL_CPUS|BPF_NOEXISTwith-EINVAL. Also theBPF_F_CPU|BPF_NOEXISTwere accepted. After the patchBPF_F_ALL_CPUS|BPF_EXISTis accepted andBPF_F_ALL_CPUS|BPF_NOEXISTandBPF_F_CPU|BPF_NOEXISTresult in-EEXIST.
Quoted 2 lines
Fixes 8eb76cb03f0f ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_array maps")Signed-off-by Masoud Aghasi <[email protected]>
lgtm,
Quoted 1 line
[...]
PATCH v5 2/3 bpf: Fix incorrect handling of user flags by percpu hash map updates
In reply to Masoud AghasiFor BPF_MAP_TYPE_PERCPU_HASH and BPF_MAP_TYPE_LRU_PERCPU_HASH maps, htab_map_check_update_flags() and check_flags() are not considering the possibility of a combination of (BPF_NOEXIST, BPF_EXIST) flags with (BPF_F_CPU, BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST, BPF_EXIST) flags to lose their effect in some cases.
For example, when using (BPF_F_CPU | BPF_EXIST) or (BPF_F_CPU | BPF_NOEXIST) flag combinations with bpf_map_update_elem() on a percpu hash map, the BPF_EXIST flag does not prevent new insertions as expected and BPF_NOEXIST flag does not prevent modification of existing entries as expected.
This patch fixes the bug by adding proper flag validations and checks. Before this patch, htab_map_check_update_flags() rejected (BPF_F_ALL_CPUS | BPF_EXIST) and (BPF_F_ALL_CPUS | BPF_NOEXIST) with -EINVAL. After the patch those combinations are accepted.
This patch also starts returning -EINVAL for (BPF_EXIST | BPF_NOEXIST), with or without BPF_F_CPU, which was previously accepted.
--- kernel/bpf/hashtab.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
kernel/bpf/hashtab.c +6 -4
@@ -1196,11 +1196,11 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key,static int check_flags (struct bpf_htab *htab,struct htab_elem *l_old,u64 map_flags) { -if (l_old && (map_flags & ~BPF_F_LOCK ) ==BPF_NOEXIST ) +if (l_old && (map_flags &BPF_NOEXIST ))/* elem already exists */ return -EEXIST ; -if (!l_old && (map_flags & ~BPF_F_LOCK ) ==BPF_EXIST ) +if (!l_old && (map_flags &BPF_EXIST ))/* elem doesn't exist, cannot update it */ return -ENOENT ;
@@ -1383,9 +1383,11 @@ static long htab_lru_map_update_elem(struct bpf_map *map, void *key, void *valuestatic int htab_map_check_update_flags (bool onallcpus,u64 map_flags) { +if (unlikely ((map_flags &BPF_EXIST_FLAGS ) ==BPF_EXIST_FLAGS )) +return -EINVAL ;if (unlikely (!onallcpus && map_flags >BPF_EXIST ))return -EINVAL ; -if (unlikely (onallcpus && ((map_flags &BPF_F_LOCK ) || (u32 )map_flags >BPF_F_ALL_CPUS ))) +if (unlikely (onallcpus && ((u32 )map_flags & ~(BPF_EXIST_FLAGS |BPF_CPU_FLAGS ))))return -EINVAL ;return 0 ; }
@@ -1483,7 +1485,7 @@ static long __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key,* to remove older elem from htab and this removal * operation will need a bucket lock. */ -if (map_flags !=BPF_EXIST ) { +if (!(map_flags &BPF_EXIST )) { l_new =prealloc_lru_pop (htab, key, hash);if (!l_new)return -ENOMEM ;
Signature · 1 line
2.47.3
Re: [PATCH bpf v5 2/3] bpf: Fix incorrect handling of user flags by percpu hash map updates
Quoted 17 lines · Masoud Aghasi
For
BPF_MAP_TYPE_PERCPU_HASHandBPF_MAP_TYPE_LRU_PERCPU_HASHmaps,htab_map_check_update_flags()andcheck_flags()are not considering the possibility of a combination of (BPF_NOEXIST,BPF_EXIST) flags with (BPF_F_CPU,BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST,BPF_EXIST) flags to lose their effect in some cases.For example, when using (
BPF_F_CPU|BPF_EXIST) or (BPF_F_CPU|BPF_NOEXIST) flag combinations withbpf_map_update_elem()on a percpu hash map, theBPF_EXISTflag does not prevent new insertions as expected andBPF_NOEXISTflag does not prevent modification of existing entries as expected.This patch fixes the bug by adding proper flag validations and checks. Before this patch,
htab_map_check_update_flags()rejected (BPF_F_ALL_CPUS|BPF_EXIST) and (BPF_F_ALL_CPUS|BPF_NOEXIST) with-EINVAL. After the patch those combinations are accepted.
Quoted 5 lines
This patch also starts returning
-EINVALfor (BPF_EXIST|BPF_NOEXIST), with or withoutBPF_F_CPU, which was previously accepted.Fixes c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps")Signed-off-by Masoud Aghasi <[email protected]>
lgtm,
Quoted 1 line
[...]
PATCH v5 3/3 selftests/bpf: add tests for percpu map flags combination
In reply to Masoud AghasiAll possible combinations of (BPF_EXIST, BPF_NOEXIST) and (BPF_F_ALL_CPUS, BPF_F_CPU) flags are covered for all percpu map types.
As BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE does not support BPF_NOEXIST and also in order to reduce the amount of code duplicates, I added its new test scenarios to the existing cpu_flag_percpu_cgroup_storage test.
But for other percpu map types, I added new tests to be able to exercise all flag combinations and the edge cases such as percpu LRU unnecessary deletion issue.
--- .../selftests/bpf/prog_tests/percpu_alloc.c | 249 ++++++++++++++++++ 1 file changed, 249 insertions(+)
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +249 -0
@@ -449,6 +449,90 @@ static void test_lru_percpu_hash_cpu_flag_create(void)test_percpu_map_cpu_flag_create (BPF_MAP_TYPE_LRU_PERCPU_HASH ,0 ); } +static bool verify_u32_map_value_all_cpus (int map_fd,void *key,u32 *values, +u32 expected_value,int nr_cpus) +{ +int i, err; + + err =bpf_map_lookup_elem (map_fd, key, values); +if (!ASSERT_OK (err,"bpf_map_lookup_elem all_cpus" )) +return false; + +for (i =0 ; i < nr_cpus ; i++) { +if (!ASSERT_EQ (values[i *2 ], expected_value,"bpf_map_lookup_elem value all_cpus" )) +return false; + } + +return true; +} + +static bool verify_u32_map_value_first_cpu (int map_fd,void *key,u32 *values, +u32 expected_value,int nr_cpus) +{ +int i, err; + + err =bpf_map_lookup_elem (map_fd, key, values); +if (!ASSERT_OK (err,"bpf_map_lookup_elem cpu" )) +return false; + +for (i =1 ; i < nr_cpus ; i++) { +if (!ASSERT_NEQ (values[i *2 ], expected_value,"bpf_map_lookup_elem value cpu" )) +return false; + } + +return ASSERT_EQ (values[0 ], expected_value,"bpf_map_lookup_elem value cpu" ); +} + +static void test_percpu_cgroup_storage_flags_combination (struct bpf_map *map,int nr_cpus, +struct bpf_cgroup_storage_key *key) +{ +size_t value_sz =sizeof (u32), key_sz, elem_sz; +u32 *values, test_value =0xAABBCCDD ; +int err, map_fd; +u64 flags; + + key_sz =sizeof (*key); + elem_sz =roundup (value_sz,8 ); + map_fd =bpf_map__fd (map); + values =calloc (nr_cpus, elem_sz); +if (!ASSERT_OK_PTR (values,"calloc values" )) +return ; + + flags =BPF_NOEXIST |BPF_EXIST ; + err =bpf_map__update_elem (map, key, key_sz, values, elem_sz * nr_cpus, flags); +if (!ASSERT_EQ (err, -EINVAL ,"bpf_map__update_elem noexist|exist" )) + goto out; + + flags =BPF_F_ALL_CPUS |BPF_NOEXIST ; + err =bpf_map__update_elem (map, key, key_sz, values, value_sz, flags); +if (!ASSERT_EQ (err, -EINVAL ,"bpf_map__update_elem all_cpus|noexist" )) + goto out; + + flags =BPF_F_CPU |BPF_NOEXIST ; + err =bpf_map__update_elem (map, key, key_sz, values, value_sz, flags); +if (!ASSERT_EQ (err, -EINVAL ,"bpf_map__update_elem cpu|noexist" )) + goto out; + + flags =BPF_F_ALL_CPUS |BPF_EXIST ; + values[0 ] = test_value; + err =bpf_map__update_elem (map, key, key_sz, values, value_sz, flags); +if (!ASSERT_OK (err,"bpf_map__update_elem all_cpus|exist" )) + goto out; + +if (!verify_u32_map_value_all_cpus (map_fd, key, values, test_value, nr_cpus)) + goto out; + + flags =BPF_F_CPU |BPF_EXIST ; + values[0 ] = --test_value; + err =bpf_map__update_elem (map, key, key_sz, values, value_sz, flags); +if (!ASSERT_OK (err,"bpf_map__update_elem cpu|exist" )) + goto out; + +verify_u32_map_value_first_cpu (map_fd, key, values, test_value, nr_cpus); +out: +free (values); +} +static void test_percpu_cgroup_storage_cpu_flag (void ) {struct percpu_alloc_array *skel =NULL ;
@@ -489,6 +573,7 @@ static void test_percpu_cgroup_storage_cpu_flag(void) goto out;test_percpu_map_op_cpu_flag (map, &key,sizeof (key),1 , nr_cpus, false); +test_percpu_cgroup_storage_flags_combination (map, nr_cpus, &key); out:bpf_prog_detach2 (-1 , cgroup,BPF_CGROUP_INET_EGRESS );close (cgroup);
@@ -537,6 +622,164 @@ static void test_hash_cpu_flag(void)test_map_op_cpu_flag (BPF_MAP_TYPE_HASH ); } +static void test_percpu_map_flags_combination (enum bpf_map_type map_type) +{ +u32 max_entries =3 , key =0 , first_value =0xAABBCCDD , test_value = first_value; +size_t value_sz =sizeof (u32), elem_sz; +int err, map_fd, nr_cpus; +bool is_hash_map; +u32 *values; +u64 flags; + + is_hash_map = (map_type ==BPF_MAP_TYPE_PERCPU_HASH || + map_type ==BPF_MAP_TYPE_LRU_PERCPU_HASH ); + nr_cpus =libbpf_num_possible_cpus (); +if (!ASSERT_GT (nr_cpus,0 ,"libbpf_num_possible_cpus" )) +return ; + + elem_sz =roundup (value_sz,8 ); + values =calloc (nr_cpus, elem_sz); +if (!ASSERT_OK_PTR (values,"calloc values" )) +return ; + + map_fd =bpf_map_create (map_type,"test_flags_combination_map" , +sizeof (u32), value_sz, max_entries,NULL ); +if (!ASSERT_GE (map_fd,0 ,"bpf_map_create" )) { +free (values); +return ; + } + + flags =BPF_NOEXIST |BPF_EXIST ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_EQ (err, -EINVAL ,"bpf_map_update_elem noexist|exist" )) + goto out; + + flags =BPF_F_ALL_CPUS |BPF_NOEXIST ; + values[0 ] = test_value; + key =0 ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (is_hash_map) { +if (!ASSERT_OK (err,"bpf_map_update_elem all_cpus|noexist" )) + goto out; + +if (!verify_u32_map_value_all_cpus (map_fd, &key, values, test_value, nr_cpus)) + goto out; + + err =bpf_map_update_elem (map_fd, &key, values, flags); + } +if (!ASSERT_EQ (err, -EEXIST ,"bpf_map_update_elem all_cpus|noexist" )) + goto out; + + flags =BPF_F_CPU |BPF_NOEXIST ; + values[0 ] = --test_value; + key =1 ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (is_hash_map) { +if (!ASSERT_OK (err,"bpf_map_update_elem cpu|noexist" )) + goto out; + +if (!verify_u32_map_value_first_cpu (map_fd, &key, values, test_value, nr_cpus)) + goto out; + + err =bpf_map_update_elem (map_fd, &key, values, flags); + } +if (!ASSERT_EQ (err, -EEXIST ,"bpf_map_update_elem cpu|noexist" )) + goto out; + + flags =BPF_F_ALL_CPUS |BPF_EXIST ; + values[0 ] = --test_value; + key =2 ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (is_hash_map) { +if (!ASSERT_EQ (err, -ENOENT ,"bpf_map_update_elem all_cpus|exist" )) + goto out; + }else { +if (!ASSERT_OK (err,"bpf_map_update_elem all_cpus|exist" )) + goto out; + +if (!verify_u32_map_value_all_cpus (map_fd, &key, values, test_value, nr_cpus)) + goto out; + } + + flags =BPF_F_CPU |BPF_EXIST ; + values[0 ] = --test_value; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (is_hash_map) { +if (!ASSERT_EQ (err, -ENOENT ,"bpf_map_update_elem cpu|exist" )) + goto out; + }else { +if (!ASSERT_OK (err,"bpf_map_update_elem cpu|exist" )) + goto out; + +if (!verify_u32_map_value_first_cpu (map_fd, &key, values, test_value, nr_cpus)) + goto out; + } + +if (map_type !=BPF_MAP_TYPE_LRU_PERCPU_HASH ) + goto out; + +/* Percpu LRU hash map should not delete old entries unnecessarily */ + flags =BPF_F_ALL_CPUS |BPF_NOEXIST ; + key =2 ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_OK (err,"bpf_map_update_elem unnecessary_deletion" )) + goto out; + + flags =BPF_F_ALL_CPUS |BPF_EXIST ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_OK (err,"bpf_map_update_elem unnecessary_deletion" )) + goto out; + + key =0 ; +verify_u32_map_value_all_cpus (map_fd, &key, values, first_value, nr_cpus); + +out: +close (map_fd); +free (values); +} + +static bool pin_current_cpu (cpu_set_t *old_mask) +{ +cpu_set_t new_mask; +int err, cpu; + + err =sched_getaffinity (0 ,sizeof (*old_mask), old_mask); +if (!ASSERT_OK (err,"sched_getaffinity" )) +return false; + + cpu =sched_getcpu (); +if (!ASSERT_GE (cpu,0 ,"sched_getcpu" )) +return false; + +CPU_ZERO (&new_mask); +CPU_SET (cpu, &new_mask); + + err =sched_setaffinity (0 ,sizeof (new_mask), &new_mask); +return ASSERT_OK (err,"sched_setaffinity" ); +} + +static void test_percpu_hash_flags_combination (void ) +{ +test_percpu_map_flags_combination (BPF_MAP_TYPE_PERCPU_HASH ); +} + +static void test_lru_percpu_hash_flags_combination (void ) +{ +cpu_set_t old_mask; + +if (!pin_current_cpu (&old_mask)) +return ; + +test_percpu_map_flags_combination (BPF_MAP_TYPE_LRU_PERCPU_HASH ); + +sched_setaffinity (0 ,sizeof (old_mask), &old_mask); +} + +static void test_percpu_array_flags_combination (void ) +{ +test_percpu_map_flags_combination (BPF_MAP_TYPE_PERCPU_ARRAY ); +} +void test_percpu_alloc (void ) {if (test__start_subtest ("array" ))
@@ -565,4 +808,10 @@ void test_percpu_alloc(void)test_array_cpu_flag ();if (test__start_subtest ("cpu_flag_hash" ))test_hash_cpu_flag (); +if (test__start_subtest ("cpu_flag_combination_percpu_hash" )) +test_percpu_hash_flags_combination (); +if (test__start_subtest ("cpu_flag_combination_lru_percpu_hash" )) +test_lru_percpu_hash_flags_combination (); +if (test__start_subtest ("cpu_flag_combination_percpu_array" )) +test_percpu_array_flags_combination (); }
Signature · 1 line
2.47.3
Re: [PATCH bpf v5 3/3] selftests/bpf: add tests for percpu map flags combination
Quoted 21 lines · Masoud Aghasi
All possible combinations of (
BPF_EXIST,BPF_NOEXIST) and (BPF_F_ALL_CPUS,BPF_F_CPU) flags are covered for all percpu map types.As
BPF_MAP_TYPE_PERCPU_CGROUP_STORAGEdoes not supportBPF_NOEXISTand also in order to reduce the amount of code duplicates, I added its new test scenarios to the existingcpu_flag_percpu_cgroup_storagetest.But for other percpu map types, I added new tests to be able to exercise all flag combinations and the edge cases such as percpu LRU unnecessary deletion issue.
Signed-off-by Masoud Aghasi <[email protected]>--- .../selftests/bpf/prog_tests/percpu_alloc.c | 249 ++++++++++++++++++ 1 file changed, 249 insertions(+)tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +0
-0
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c static void test_lru_percpu_hash_cpu_flag_create(void) · line 453
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +2
-0@@ -449,6 +449,90 @@ static void test_lru_percpu_hash_cpu_flag_create(void)test_percpu_map_cpu_flag_create (BPF_MAP_TYPE_LRU_PERCPU_HASH ,0 ); } +static bool verify_u32_map_value_all_cpus (int map_fd,void *key,u32 *values, +u32 expected_value,int nr_cpus)
NIT: 'u32' seems unnecessary in the function name.
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c static void test_lru_percpu_hash_cpu_flag_create(void) · line 462
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +9
-0+{ +int i, err; + + err =bpf_map_lookup_elem (map_fd, key, values); +if (!ASSERT_OK (err,"bpf_map_lookup_elem all_cpus" )) +return false; + +for (i =0 ; i < nr_cpus ; i++) { +if (!ASSERT_EQ (values[i *2 ], expected_value,"bpf_map_lookup_elem value all_cpus" ))
'i * 2' looks hard to understand.
Should pass 'void *values, size_t elem_sz', then get value by '*(u32 *)(value + elem_sz * i)'
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c static void test_lru_percpu_hash_cpu_flag_create(void) · line 470
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +8
-0+return false; + } + +return true; +} + +static bool verify_u32_map_value_first_cpu (int map_fd,void *key,u32 *values, +u32 expected_value,int nr_cpus)
Ditto.
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c static void test_lru_percpu_hash_cpu_flag_create(void) · line 476
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +6
-0+{ +int i, err; + + err =bpf_map_lookup_elem (map_fd, key, values); +if (!ASSERT_OK (err,"bpf_map_lookup_elem cpu" )) +return false;
Can ASSERT_EQ() the first slot here instead of the last? Just for readability.
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c static void test_lru_percpu_hash_cpu_flag_create(void) · line 479
Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +3
-0+ +for (i =1 ; i < nr_cpus ; i++) { +if (!ASSERT_NEQ (values[i *2 ], expected_value,"bpf_map_lookup_elem value cpu" ))
Ditto.
Quoted 6 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +6
-0+return false; + } + +return ASSERT_EQ (values[0 ], expected_value,"bpf_map_lookup_elem value cpu" ); +} +
[...]
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c static void test_hash_cpu_flag(void) · line 725
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +9
-0+ +if (map_type !=BPF_MAP_TYPE_LRU_PERCPU_HASH ) + goto out; + +/* Percpu LRU hash map should not delete old entries unnecessarily */ + flags =BPF_F_ALL_CPUS |BPF_NOEXIST ; + key =2 ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_OK (err,"bpf_map_update_elem unnecessary_deletion" ))
NIT: unnecessary_deletion -> lru_percpu_hash all_cpus|noexist
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c static void test_hash_cpu_flag(void) · line 725
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +5
-0+ goto out; + + flags =BPF_F_ALL_CPUS |BPF_EXIST ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_OK (err,"bpf_map_update_elem unnecessary_deletion" ))
NIT: unnecessary_deletion -> lru_percpu_hash all_cpus|exist
Thanks,
Leon
Quoted 2 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +1
-0+ goto out;
Re: [PATCH bpf v5 3/3] selftests/bpf: add tests for percpu map flags combination
Quoted 2 lines · Leon Hwang
NIT: 'u32' seems unnecessary in the function name.
[...]
Quoted 6 lines
'i * 2' looks hard to understand.
Should pass 'void *values,
size_telem_sz', then get value by '*(u32 *)(value +elem_sz* i)'
[...]
Quoted 4 lines
Can
ASSERT_EQ()the first slot here instead of the last? Just for readability.
[...]
Quoted 3 lines
NIT:
unnecessary_deletion->lru_percpu_hashall_cpus|noexist
[...]
Quoted 3 lines
NIT:
unnecessary_deletion->lru_percpu_hashall_cpus|exist
[...]
Thanks, will include them in v6.