bpf: Fix incorrect handling of user flags by percpu map updates
Subsystems (4)
v6 cover bpf: Fix incorrect handling of user flags by percpu map updates
For BPF_MAP_TYPE_PERCPU_ARRAY map, bpf_percpu_array_update() is not considering the possibility of a combination of (BPF_NOEXIST, BPF_EXIST) flags with (BPF_F_CPU, BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST, BPF_EXIST) flags to lose their effect in some cases.
For BPF_MAP_TYPE_PERCPU_HASH and BPF_MAP_TYPE_LRU_PERCPU_HASH maps, htab_map_check_update_flags() and check_flags() are not considering the possibility of a combination of (BPF_NOEXIST, BPF_EXIST) flags with (BPF_F_CPU, BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST, BPF_EXIST) flags to lose their effect in some cases.
For example, when using (BPF_F_CPU | BPF_EXIST) flag combination with bpf_map_update_elem() on a BPF_MAP_TYPE_PERCPU_HASH map, the BPF_EXIST flag does not prevent new insertions as expected.
This series fixes the bug by adding proper flag validations and checks and adds regression tests.
V6 changes:
- Improve the style and structure of the selftest
V5 changes: https://lore.kernel.org/bpf/[email protected]/T/
- Simplify flag validations by introducing new macros in bpf.h
- Add helpers to verify the values after updates in the new tests
- Improve the style and structure of the new tests
V4 changes: https://lore.kernel.org/bpf/[email protected]/T/
- Edit selftest to use proper value_sz in the new cgroup map test
- Edit selftest to pin the current pid to the current cpu for LRU map
- Edit commit messages to include user-visible changes
V3 changes: https://lore.kernel.org/bpf/[email protected]/T/
- Split the fix into two separate patches for array map and hash maps
- Extend the selftest to cover all percpu map types
- Extend the selftest to cover all applicable flag combinations
- Extend the selftest to cover the unnecessary delete issue of LRU map
V2 changes: https://lore.kernel.org/bpf/[email protected]/T/
- Fix a BPF_EXIST flag check in __htab_lru_percpu_map_update_elem()
- Add additional test for BPF_MAP_TYPE_LRU_PERCPU_HASH map
- Add check for BPF_EXIST, BPF_NOEXIST combination in percpu array map
V1: https://lore.kernel.org/bpf/[email protected]/T/
Masoud Aghasi (3):
bpf: Fix incorrect handling of user flags in bpf_percpu_array_update
bpf: Fix incorrect handling of user flags by percpu hash map updates selftests/bpf: add tests for percpu map flags combination
include/linux/bpf.h | 3 + kernel/bpf/arraymap.c | 6 +- kernel/bpf/hashtab.c | 10 +- .../selftests/bpf/prog_tests/percpu_alloc.c | 254 ++++++++++++++++++ 4 files changed, 266 insertions(+), 7 deletions(-)
Signature · 1 line
2.47.3
PATCH v6 1/3 bpf: Fix incorrect handling of user flags in bpf_percpu_array_update
For BPF_MAP_TYPE_PERCPU_ARRAY map, bpf_percpu_array_update() is not considering the possibility of a combination of (BPF_NOEXIST, BPF_EXIST) flags with (BPF_F_CPU, BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST, BPF_EXIST) flags to lose their effect in some cases.
For example, using the (BPF_F_ALL_CPUS | BPF_EXIST) flag combination with bpf_map_update_elem() results in an incorrect EINVAL error response, even though the flag combination is valid.
This patch fixes the bug by adding proper flag validations and checks. Before this patch, bpf_percpu_array_update() rejected BPF_F_ALL_CPUS | BPF_EXIST and BPF_F_ALL_CPUS | BPF_NOEXIST with -EINVAL. Also the BPF_F_CPU | BPF_NOEXIST were accepted. After the patch BPF_F_ALL_CPUS | BPF_EXIST is accepted and BPF_F_ALL_CPUS | BPF_NOEXIST and BPF_F_CPU | BPF_NOEXIST result in -EEXIST.
This patch also starts returning -EINVAL for (BPF_EXIST | BPF_NOEXIST), with or without BPF_F_CPU, which was previously accepted.
--- include/linux/bpf.h | 3 +++ kernel/bpf/arraymap.c | 6 +++--- 2 files changed, 6 insertions(+), 3 deletions(-)
include/linux/bpf.h +3 -0
@@ -4322,6 +4322,9 @@ static inline bool bpf_map_is_percpu_map(enum bpf_map_type map_type) } } +#define BPF_EXIST_FLAGS (BPF_EXIST | BPF_NOEXIST) +#define BPF_CPU_FLAGS (BPF_F_CPU | BPF_F_ALL_CPUS) +static inline int bpf_map_check_op_flags (struct bpf_map *map,u64 flags,u64 allowed_flags) {u32 cpu;
kernel/bpf/arraymap.c +3 -3
@@ -438,15 +438,15 @@ int bpf_percpu_array_update(struct bpf_map *map, void *key, void *value,u32 size;int cpu, off =0 ; -if (unlikely ((map_flags &BPF_F_LOCK ) || (u32 )map_flags >BPF_F_ALL_CPUS )) -/* unknown flags */ +if (unlikely (((map_flags &BPF_EXIST_FLAGS ) ==BPF_EXIST_FLAGS ) || + ((u32 )map_flags & ~(BPF_EXIST_FLAGS |BPF_CPU_FLAGS ))))return -EINVAL ;if (unlikely (index >= array->map.max_entries))/* all elements were pre-allocated, cannot insert a new one */ return -E2BIG ; -if (unlikely (map_flags ==BPF_NOEXIST )) +if (unlikely (map_flags &BPF_NOEXIST ))/* all elements already exist */ return -EEXIST ;
Signature · 1 line
2.47.3
Re: [PATCH bpf v6 1/3] bpf: Fix incorrect handling of user flags in bpf_percpu_array_update
Quoted 18 lines · Masoud Aghasi
For
BPF_MAP_TYPE_PERCPU_ARRAYmap,bpf_percpu_array_update()is not considering the possibility of a combination of (BPF_NOEXIST,BPF_EXIST) flags with (BPF_F_CPU,BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST,BPF_EXIST) flags to lose their effect in some cases.For example, using the (
BPF_F_ALL_CPUS|BPF_EXIST) flag combination withbpf_map_update_elem()results in an incorrect EINVAL error response, even though the flag combination is valid.This patch fixes the bug by adding proper flag validations and checks. Before this patch,
bpf_percpu_array_update()rejectedBPF_F_ALL_CPUS|BPF_EXISTandBPF_F_ALL_CPUS|BPF_NOEXISTwith-EINVAL. Also theBPF_F_CPU|BPF_NOEXISTwere accepted. After the patchBPF_F_ALL_CPUS|BPF_EXISTis accepted andBPF_F_ALL_CPUS|BPF_NOEXISTandBPF_F_CPU|BPF_NOEXISTresult in-EEXIST.
Quoted 5 lines
This patch also starts returning
-EINVALfor (BPF_EXIST|BPF_NOEXIST), with or withoutBPF_F_CPU, which was previously accepted.Fixes 8eb76cb03f0f ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_array maps")Signed-off-by Masoud Aghasi <[email protected]>
Should carry my Acked-by in v5.
Quoted 1 line
[...]
PATCH v6 2/3 bpf: Fix incorrect handling of user flags by percpu hash map updates
In reply to Masoud AghasiFor BPF_MAP_TYPE_PERCPU_HASH and BPF_MAP_TYPE_LRU_PERCPU_HASH maps, htab_map_check_update_flags() and check_flags() are not considering the possibility of a combination of (BPF_NOEXIST, BPF_EXIST) flags with (BPF_F_CPU, BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST, BPF_EXIST) flags to lose their effect in some cases.
For example, when using (BPF_F_CPU | BPF_EXIST) or (BPF_F_CPU | BPF_NOEXIST) flag combinations with bpf_map_update_elem() on a percpu hash map, the BPF_EXIST flag does not prevent new insertions as expected and BPF_NOEXIST flag does not prevent modification of existing entries as expected.
This patch fixes the bug by adding proper flag validations and checks. Before this patch, htab_map_check_update_flags() rejected (BPF_F_ALL_CPUS | BPF_EXIST) and (BPF_F_ALL_CPUS | BPF_NOEXIST) with -EINVAL. After the patch those combinations are accepted.
This patch also starts returning -EINVAL for (BPF_EXIST | BPF_NOEXIST), with or without BPF_F_CPU, which was previously accepted.
--- kernel/bpf/hashtab.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
kernel/bpf/hashtab.c +6 -4
@@ -1196,11 +1196,11 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key,static int check_flags (struct bpf_htab *htab,struct htab_elem *l_old,u64 map_flags) { -if (l_old && (map_flags & ~BPF_F_LOCK ) ==BPF_NOEXIST ) +if (l_old && (map_flags &BPF_NOEXIST ))/* elem already exists */ return -EEXIST ; -if (!l_old && (map_flags & ~BPF_F_LOCK ) ==BPF_EXIST ) +if (!l_old && (map_flags &BPF_EXIST ))/* elem doesn't exist, cannot update it */ return -ENOENT ;
@@ -1383,9 +1383,11 @@ static long htab_lru_map_update_elem(struct bpf_map *map, void *key, void *valuestatic int htab_map_check_update_flags (bool onallcpus,u64 map_flags) { +if (unlikely ((map_flags &BPF_EXIST_FLAGS ) ==BPF_EXIST_FLAGS )) +return -EINVAL ;if (unlikely (!onallcpus && map_flags >BPF_EXIST ))return -EINVAL ; -if (unlikely (onallcpus && ((map_flags &BPF_F_LOCK ) || (u32 )map_flags >BPF_F_ALL_CPUS ))) +if (unlikely (onallcpus && ((u32 )map_flags & ~(BPF_EXIST_FLAGS |BPF_CPU_FLAGS ))))return -EINVAL ;return 0 ; }
@@ -1483,7 +1485,7 @@ static long __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key,* to remove older elem from htab and this removal * operation will need a bucket lock. */ -if (map_flags !=BPF_EXIST ) { +if (!(map_flags &BPF_EXIST )) { l_new =prealloc_lru_pop (htab, key, hash);if (!l_new)return -ENOMEM ;
Signature · 1 line
2.47.3
Re: [PATCH bpf v6 2/3] bpf: Fix incorrect handling of user flags by percpu hash map updates
Quoted 17 lines · Masoud Aghasi
For
BPF_MAP_TYPE_PERCPU_HASHandBPF_MAP_TYPE_LRU_PERCPU_HASHmaps,htab_map_check_update_flags()andcheck_flags()are not considering the possibility of a combination of (BPF_NOEXIST,BPF_EXIST) flags with (BPF_F_CPU,BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST,BPF_EXIST) flags to lose their effect in some cases.For example, when using (
BPF_F_CPU|BPF_EXIST) or (BPF_F_CPU|BPF_NOEXIST) flag combinations withbpf_map_update_elem()on a percpu hash map, theBPF_EXISTflag does not prevent new insertions as expected andBPF_NOEXISTflag does not prevent modification of existing entries as expected.This patch fixes the bug by adding proper flag validations and checks. Before this patch,
htab_map_check_update_flags()rejected (BPF_F_ALL_CPUS|BPF_EXIST) and (BPF_F_ALL_CPUS|BPF_NOEXIST) with-EINVAL. After the patch those combinations are accepted.
Quoted 5 lines
This patch also starts returning
-EINVALfor (BPF_EXIST|BPF_NOEXIST), with or withoutBPF_F_CPU, which was previously accepted.Fixes c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps")Signed-off-by Masoud Aghasi <[email protected]>
Should carry my Acked-by in v5.
Quoted 1 line
[...]
PATCH v6 3/3 selftests/bpf: add tests for percpu map flags combination
In reply to Masoud AghasiAll possible combinations of (BPF_EXIST, BPF_NOEXIST) and (BPF_F_ALL_CPUS, BPF_F_CPU) flags are covered for all percpu map types.
As BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE does not support BPF_NOEXIST and also in order to reduce the amount of code duplicates, I added its new test scenarios to the existing cpu_flag_percpu_cgroup_storage test.
But for other percpu map types, I added new tests to be able to exercise all flag combinations and the edge cases such as percpu LRU unnecessary deletion issue.
--- .../selftests/bpf/prog_tests/percpu_alloc.c | 254 ++++++++++++++++++ 1 file changed, 254 insertions(+)
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +254 -0
@@ -449,6 +449,95 @@ static void test_lru_percpu_hash_cpu_flag_create(void)test_percpu_map_cpu_flag_create (BPF_MAP_TYPE_LRU_PERCPU_HASH ,0 ); } +static bool verify_map_value_all_cpus (int map_fd,void *key,void *values,size_t elem_sz, +u32 expected_value,int nr_cpus) +{ +int i, err; + + err =bpf_map_lookup_elem (map_fd, key, values); +if (!ASSERT_OK (err,"bpf_map_lookup_elem all_cpus" )) +return false; + +for (i =0 ; i < nr_cpus ; i++) { +if (!ASSERT_EQ (*(u32 *)(values + elem_sz * i), expected_value, +"bpf_map_lookup_elem value all_cpus" )) +return false; + } + +return true; +} + +static bool verify_map_value_first_cpu (int map_fd,void *key,void *values,size_t elem_sz, +u32 expected_value,int nr_cpus) +{ +int i, err; + + err =bpf_map_lookup_elem (map_fd, key, values); +if (!ASSERT_OK (err,"bpf_map_lookup_elem cpu" )) +return false; + +if (!ASSERT_EQ (*(u32 *)values, expected_value,"bpf_map_lookup_elem value cpu" )) +return false; + +for (i =1 ; i < nr_cpus ; i++) { +if (!ASSERT_NEQ (*(u32 *)(values + elem_sz * i), expected_value, +"bpf_map_lookup_elem value cpu" )) +return false; + } + +return true; +} + +static void test_percpu_cgroup_storage_flags_combination (struct bpf_map *map,int nr_cpus, +struct bpf_cgroup_storage_key *key) +{ +size_t value_sz =sizeof (u32), key_sz, elem_sz; +u32 *values, test_value =0xAABBCCDD ; +int err, map_fd; +u64 flags; + + key_sz =sizeof (*key); + elem_sz =roundup (value_sz,8 ); + map_fd =bpf_map__fd (map); + values =calloc (nr_cpus, elem_sz); +if (!ASSERT_OK_PTR (values,"calloc values" )) +return ; + + flags =BPF_NOEXIST |BPF_EXIST ; + err =bpf_map__update_elem (map, key, key_sz, values, elem_sz * nr_cpus, flags); +if (!ASSERT_EQ (err, -EINVAL ,"bpf_map__update_elem noexist|exist" )) + goto out; + + flags =BPF_F_ALL_CPUS |BPF_NOEXIST ; + err =bpf_map__update_elem (map, key, key_sz, values, value_sz, flags); +if (!ASSERT_EQ (err, -EINVAL ,"bpf_map__update_elem all_cpus|noexist" )) + goto out; + + flags =BPF_F_CPU |BPF_NOEXIST ; + err =bpf_map__update_elem (map, key, key_sz, values, value_sz, flags); +if (!ASSERT_EQ (err, -EINVAL ,"bpf_map__update_elem cpu|noexist" )) + goto out; + + flags =BPF_F_ALL_CPUS |BPF_EXIST ; + values[0 ] = test_value; + err =bpf_map__update_elem (map, key, key_sz, values, value_sz, flags); +if (!ASSERT_OK (err,"bpf_map__update_elem all_cpus|exist" )) + goto out; + +if (!verify_map_value_all_cpus (map_fd, key, values, elem_sz, test_value, nr_cpus)) + goto out; + + flags =BPF_F_CPU |BPF_EXIST ; + values[0 ] = --test_value; + err =bpf_map__update_elem (map, key, key_sz, values, value_sz, flags); +if (!ASSERT_OK (err,"bpf_map__update_elem cpu|exist" )) + goto out; + +verify_map_value_first_cpu (map_fd, key, values, elem_sz, test_value, nr_cpus); +out: +free (values); +} +static void test_percpu_cgroup_storage_cpu_flag (void ) {struct percpu_alloc_array *skel =NULL ;
@@ -489,6 +578,7 @@ static void test_percpu_cgroup_storage_cpu_flag(void) goto out;test_percpu_map_op_cpu_flag (map, &key,sizeof (key),1 , nr_cpus, false); +test_percpu_cgroup_storage_flags_combination (map, nr_cpus, &key); out:bpf_prog_detach2 (-1 , cgroup,BPF_CGROUP_INET_EGRESS );close (cgroup);
@@ -537,6 +627,164 @@ static void test_hash_cpu_flag(void)test_map_op_cpu_flag (BPF_MAP_TYPE_HASH ); } +static void test_percpu_map_flags_combination (enum bpf_map_type map_type) +{ +u32 max_entries =3 , key =0 , first_value =0xAABBCCDD , test_value = first_value; +size_t value_sz =sizeof (u32), elem_sz; +int err, map_fd, nr_cpus; +bool is_hash_map; +u32 *values; +u64 flags; + + is_hash_map = (map_type ==BPF_MAP_TYPE_PERCPU_HASH || + map_type ==BPF_MAP_TYPE_LRU_PERCPU_HASH ); + nr_cpus =libbpf_num_possible_cpus (); +if (!ASSERT_GT (nr_cpus,0 ,"libbpf_num_possible_cpus" )) +return ; + + elem_sz =roundup (value_sz,8 ); + values =calloc (nr_cpus, elem_sz); +if (!ASSERT_OK_PTR (values,"calloc values" )) +return ; + + map_fd =bpf_map_create (map_type,"test_flags_combination_map" , +sizeof (u32), value_sz, max_entries,NULL ); +if (!ASSERT_GE (map_fd,0 ,"bpf_map_create" )) { +free (values); +return ; + } + + flags =BPF_NOEXIST |BPF_EXIST ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_EQ (err, -EINVAL ,"bpf_map_update_elem noexist|exist" )) + goto out; + + flags =BPF_F_ALL_CPUS |BPF_NOEXIST ; + values[0 ] = test_value; + key =0 ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (is_hash_map) { +if (!ASSERT_OK (err,"bpf_map_update_elem all_cpus|noexist" )) + goto out; + +if (!verify_map_value_all_cpus (map_fd, &key, values, elem_sz, test_value, nr_cpus)) + goto out; + + err =bpf_map_update_elem (map_fd, &key, values, flags); + } +if (!ASSERT_EQ (err, -EEXIST ,"bpf_map_update_elem all_cpus|noexist" )) + goto out; + + flags =BPF_F_CPU |BPF_NOEXIST ; + values[0 ] = --test_value; + key =1 ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (is_hash_map) { +if (!ASSERT_OK (err,"bpf_map_update_elem cpu|noexist" )) + goto out; + +if (!verify_map_value_first_cpu (map_fd, &key, values, elem_sz, test_value, nr_cpus)) + goto out; + + err =bpf_map_update_elem (map_fd, &key, values, flags); + } +if (!ASSERT_EQ (err, -EEXIST ,"bpf_map_update_elem cpu|noexist" )) + goto out; + + flags =BPF_F_ALL_CPUS |BPF_EXIST ; + values[0 ] = --test_value; + key =2 ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (is_hash_map) { +if (!ASSERT_EQ (err, -ENOENT ,"bpf_map_update_elem all_cpus|exist" )) + goto out; + }else { +if (!ASSERT_OK (err,"bpf_map_update_elem all_cpus|exist" )) + goto out; + +if (!verify_map_value_all_cpus (map_fd, &key, values, elem_sz, test_value, nr_cpus)) + goto out; + } + + flags =BPF_F_CPU |BPF_EXIST ; + values[0 ] = --test_value; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (is_hash_map) { +if (!ASSERT_EQ (err, -ENOENT ,"bpf_map_update_elem cpu|exist" )) + goto out; + }else { +if (!ASSERT_OK (err,"bpf_map_update_elem cpu|exist" )) + goto out; + +if (!verify_map_value_first_cpu (map_fd, &key, values, elem_sz, test_value, nr_cpus)) + goto out; + } + +if (map_type !=BPF_MAP_TYPE_LRU_PERCPU_HASH ) + goto out; + +/* Percpu LRU hash map should not delete old entries unnecessarily */ + flags =BPF_F_ALL_CPUS |BPF_NOEXIST ; + key =2 ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_OK (err,"bpf_map_update_elem lru_percpu_hash all_cpus|noexist" )) + goto out; + + flags =BPF_F_ALL_CPUS |BPF_EXIST ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_OK (err,"bpf_map_update_elem lru_percpu_hash all_cpus|exist" )) + goto out; + + key =0 ; +verify_map_value_all_cpus (map_fd, &key, values, elem_sz, first_value, nr_cpus); + +out: +close (map_fd); +free (values); +} + +static bool pin_current_cpu (cpu_set_t *old_mask) +{ +cpu_set_t new_mask; +int err, cpu; + + err =sched_getaffinity (0 ,sizeof (*old_mask), old_mask); +if (!ASSERT_OK (err,"sched_getaffinity" )) +return false; + + cpu =sched_getcpu (); +if (!ASSERT_GE (cpu,0 ,"sched_getcpu" )) +return false; + +CPU_ZERO (&new_mask); +CPU_SET (cpu, &new_mask); + + err =sched_setaffinity (0 ,sizeof (new_mask), &new_mask); +return ASSERT_OK (err,"sched_setaffinity" ); +} + +static void test_percpu_hash_flags_combination (void ) +{ +test_percpu_map_flags_combination (BPF_MAP_TYPE_PERCPU_HASH ); +} + +static void test_lru_percpu_hash_flags_combination (void ) +{ +cpu_set_t old_mask; + +if (!pin_current_cpu (&old_mask)) +return ; + +test_percpu_map_flags_combination (BPF_MAP_TYPE_LRU_PERCPU_HASH ); + +sched_setaffinity (0 ,sizeof (old_mask), &old_mask); +} + +static void test_percpu_array_flags_combination (void ) +{ +test_percpu_map_flags_combination (BPF_MAP_TYPE_PERCPU_ARRAY ); +} +void test_percpu_alloc (void ) {if (test__start_subtest ("array" ))
@@ -565,4 +813,10 @@ void test_percpu_alloc(void)test_array_cpu_flag ();if (test__start_subtest ("cpu_flag_hash" ))test_hash_cpu_flag (); +if (test__start_subtest ("cpu_flag_combination_percpu_hash" )) +test_percpu_hash_flags_combination (); +if (test__start_subtest ("cpu_flag_combination_lru_percpu_hash" )) +test_lru_percpu_hash_flags_combination (); +if (test__start_subtest ("cpu_flag_combination_percpu_array" )) +test_percpu_array_flags_combination (); }
Signature · 1 line
2.47.3
Re: [PATCH bpf v6 3/3] selftests/bpf: add tests for percpu map flags combination
Quoted 7 lines · Masoud Aghasi
All possible combinations of (
BPF_EXIST,BPF_NOEXIST) and (BPF_F_ALL_CPUS,BPF_F_CPU) flags are covered for all percpu map types.As
BPF_MAP_TYPE_PERCPU_CGROUP_STORAGEdoes not supportBPF_NOEXISTand also in order to reduce the amount of code duplicates, I added its new test scenarios to the existingcpu_flag_percpu_cgroup_storagetest.
Quoted 5 lines
But for other percpu map types, I added new tests to be able to exercise all flag combinations and the edge cases such as percpu LRU unnecessary deletion issue.
Signed-off-by Masoud Aghasi <[email protected]>
Two concerns below, others lgtm.
Quoted 4 lines
--- .../selftests/bpf/prog_tests/percpu_alloc.c | 254 ++++++++++++++++++ 1 file changed, 254 insertions(+)
[...]
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c static void test_hash_cpu_flag(void) · line 731
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +10
-0+ +if (map_type !=BPF_MAP_TYPE_LRU_PERCPU_HASH ) + goto out; + +/* Percpu LRU hash map should not delete old entries unnecessarily */ + flags =BPF_F_ALL_CPUS |BPF_NOEXIST ; + key =2 ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_OK (err,"bpf_map_update_elem lru_percpu_hash all_cpus|noexist" )) + goto out;
Update values[0] before update_elem(), and verify_map_value_all_cpus()?
Also update values[0] before the below update_elem().
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c static void test_hash_cpu_flag(void) · line 738
Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +7
-0+ + flags =BPF_F_ALL_CPUS |BPF_EXIST ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_OK (err,"bpf_map_update_elem lru_percpu_hash all_cpus|exist" )) + goto out; + + key =0 ;
Why do you use 'k = 0' instead of '2'?
I think we should update and verify using the same key.
Thanks,
Leon
Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +7
-0+verify_map_value_all_cpus (map_fd, &key, values, elem_sz, first_value, nr_cpus); + +out: +close (map_fd); +free (values); +} +
[...]
Re: [PATCH bpf v6 3/3] selftests/bpf: add tests for percpu map flags combination
Quoted 56 lines · Leon Hwang
Quoted 7 lines · Masoud Aghasi
All possible combinations of (
BPF_EXIST,BPF_NOEXIST) and (BPF_F_ALL_CPUS,BPF_F_CPU) flags are covered for all percpu map types.As
BPF_MAP_TYPE_PERCPU_CGROUP_STORAGEdoes not supportBPF_NOEXISTand also in order to reduce the amount of code duplicates, I added its new test scenarios to the existingcpu_flag_percpu_cgroup_storagetest.Quoted 5 lines
But for other percpu map types, I added new tests to be able to exercise all flag combinations and the edge cases such as percpu LRU unnecessary deletion issue.
Signed-off-by Masoud Aghasi <[email protected]>Two concerns below, others lgtm.
Acked-by Leon Hwang <[email protected]>Quoted 4 lines
--- .../selftests/bpf/prog_tests/percpu_alloc.c | 254 ++++++++++++++++++ 1 file changed, 254 insertions(+)[...]
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c static void test_hash_cpu_flag(void) · line 731
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +10
-0+ +if (map_type !=BPF_MAP_TYPE_LRU_PERCPU_HASH ) + goto out; + +/* Percpu LRU hash map should not delete old entries unnecessarily */ + flags =BPF_F_ALL_CPUS |BPF_NOEXIST ; + key =2 ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_OK (err,"bpf_map_update_elem lru_percpu_hash all_cpus|noexist" )) + goto out;Update values[0] before
update_elem(), andverify_map_value_all_cpus()?Also update values[0] before the below
update_elem().tools/testing/selftests/bpf/prog_tests/percpu_alloc.c static void test_hash_cpu_flag(void) · line 738
Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +7
-0+ + flags =BPF_F_ALL_CPUS |BPF_EXIST ; + err =bpf_map_update_elem (map_fd, &key, values, flags); +if (!ASSERT_OK (err,"bpf_map_update_elem lru_percpu_hash all_cpus|exist" )) + goto out; + + key =0 ;Why do you use 'k = 0' instead of '2'?
I think we should update and verify using the same key.
Thanks,
Leon
Quoted 9 lines
Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/percpu_alloc.c +7
-0+verify_map_value_all_cpus (map_fd, &key, values, elem_sz, first_value, nr_cpus); + +out: +close (map_fd); +free (values); +} +[...]
Thanks for the review.
Regarding your remaining concerns, I designed these new tests to cover all changes made in arraymap.c and hashtab.c. For example, if we revert my change to __htab_lru_percpu_map_update_elem() in hashtab.c, the last verification in this test for key '0' will fail.
It shows that the change is necessary to prevent unnecessary deletion of old keys by percpu LRU maps. In the case of this test, inserting and updating the last element (key '2') of a map of 3 entries should not cause unnecessary deletion of the first key (key '0').
And regarding not verifying the last 2 updates, I left those out because we already tested the (BPF_F_ALL_CPUS | BPF_NOEXIST) and (BPF_F_ALL_CPUS | BPF_EXIST) flag combinations for LRU maps (and other types) in previous code, so there was no need to test them again. the final part of this test is only meant to cover a specific LRU map scenario that I discussed with Alexei in V2.
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 6 lines · On Thu, 8 Oct 2026 13:07:31 +0100 you wrote:
For
BPF_MAP_TYPE_PERCPU_ARRAYmap,bpf_percpu_array_update()is not considering the possibility of a combination of (BPF_NOEXIST,BPF_EXIST) flags with (BPF_F_CPU,BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST,BPF_EXIST) flags to lose their effect in some cases.
Quoted 7 lines
For
BPF_MAP_TYPE_PERCPU_HASHandBPF_MAP_TYPE_LRU_PERCPU_HASHmaps,htab_map_check_update_flags()andcheck_flags()are not considering the possibility of a combination of (BPF_NOEXIST,BPF_EXIST) flags with (BPF_F_CPU,BPF_F_ALL_CPUS) flags. This causes the (BPF_NOEXIST,BPF_EXIST) flags to lose their effect in some cases.[...]
Here is the summary with links:
- [bpf,v6,1/3] bpf: Fix incorrect handling of user flags in bpf_percpu_array_update
https://git.kernel.org/bpf/bpf-next/c/a898ff0f3e99 - [bpf,v6,2/3] bpf: Fix incorrect handling of user flags by percpu hash map updates https://git.kernel.org/bpf/bpf-next/c/4a9b25fe8d13 - [bpf,v6,3/3] selftests/bpf: add tests for percpu map flags combination https://git.kernel.org/bpf/bpf-next/c/22392799e251
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html