Don't reinit map value in prealloc_lru_pop
Subsystems (4)
Kumar Kartikeya Dwivedi
maintainerv2 cover Don't reinit map value in prealloc_lru_pop
Fix for a bug in prelloc_lru_pop spotted while reading the code, then a test + example that checks whether it is fixed.
Changelog:
----------
v1 -> v2:
v1: https://lore.kernel.org/bpf/[email protected]
* Expand commit log to include summary of the discussion with Yonghong
* Make lru_bug selftest serial to not mess up refcount for map_kptr test
Kumar Kartikeya Dwivedi (3):
bpf: Allow calling bpf_prog_test kfuncs in tracing programs
bpf: Don't reinit map value in prealloc_lru_pop selftests/bpf: Add test for prealloc_lru_pop bug
kernel/bpf/hashtab.c | 6 +- net/bpf/test_run.c | 1 + .../selftests/bpf/prog_tests/lru_bug.c | 19 ++++++ tools/testing/selftests/bpf/progs/lru_bug.c | 67 +++++++++++++++++++ 4 files changed, 88 insertions(+), 5 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/lru_bug.c create mode 100644 tools/testing/selftests/bpf/progs/lru_bug.c
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerPATCH v2 1/3 bpf: Allow calling bpf_prog_test kfuncs in tracing programs
In addition to TC hook, enable these in tracing programs so that they can be used in selftests.
--- net/bpf/test_run.c | 1 + 1 file changed, 1 insertion(+)
net/bpf/test_run.c +1 -0
@@ -1628,6 +1628,7 @@ static int __init bpf_prog_test_run_init(void)int ret; ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_SCHED_CLS , &bpf_prog_test_kfunc_set); + ret = ret ?:register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &bpf_prog_test_kfunc_set);return ret ?:register_btf_id_dtor_kfuncs (bpf_prog_test_dtor_kfunc,ARRAY_SIZE (bpf_prog_test_dtor_kfunc),THIS_MODULE );
Signature · 1 line
2.34.1
Re: [PATCH bpf v2 1/3] bpf: Allow calling bpf_prog_test kfuncs in tracing programs
Quoted 4 lines · Kumar Kartikeya Dwivedi
In addition to TC hook, enable these in tracing programs so that they can be used in selftests.
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
Kumar Kartikeya Dwivedi
maintainerPATCH v2 2/3 bpf: Don't reinit map value in prealloc_lru_pop
In reply to Kumar Kartikeya DwivediThe LRU map that is preallocated may have its elements reused while another program holds a pointer to it from bpf_map_lookup_elem. Hence, only check_and_free_fields is appropriate when the element is being deleted, as it ensures proper synchronization against concurrent access of the map value. After that, we cannot call check_and_init_map_value again as it may rewrite bpf_spin_lock, bpf_timer, and kptr fields while they can be concurrently accessed from a BPF program.
This is safe to do as when the map entry is deleted, concurrent access is protected against by check_and_free_fields, i.e. an existing timer would be freed, and any existing kptr will be released by it. The program can create further timers and kptrs after check_and_free_fields, but they will eventually be released once the preallocated items are freed on map destruction, even if the item is never reused again. Hence, the deleted item sitting in the free list can still have resources attached to it, and they would never leak.
With spin_lock, we never touch the field at all on delete or update, as we may end up modifying the state of the lock. Since the verifier ensures that a bpf_spin_lock call is always paired with bpf_spin_unlock call, the program will eventually release the lock so that on reuse the new user of the value can take the lock.
Essentially, for the preallocated case, we must assume that the map value may always be in use by the program, even when it is sitting in the freelist, and handle things accordingly, i.e. use proper synchronization inside check_and_free_fields, and never reinitialize the special fields when it is reused on update.
--- kernel/bpf/hashtab.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
kernel/bpf/hashtab.c +1 -5
@@ -311,12 +311,8 @@ static struct htab_elem *prealloc_lru_pop(struct bpf_htab *htab, void *key,struct htab_elem *l;if (node) { -u32 key_size = htab->map.key_size; - l = container_of(node, struct htab_elem, lru_node); -memcpy (l->key, key, key_size); -check_and_init_map_value (&htab->map, - l->key +round_up (key_size,8 )); +memcpy (l->key, key, htab->map.key_size);return l; }
Signature · 1 line
2.34.1
Kumar Kartikeya Dwivedi
maintainerPATCH v2 3/3 selftests/bpf: Add test for prealloc_lru_pop bug
In reply to Kumar Kartikeya DwivediAdd a regression test to check against invalid check_and_init_map_value call inside prealloc_lru_pop.
To actually observe a kind of problem this can cause, set debug to 1 when running the test locally without the fix. Then, just observe the refcount which keeps increasing on each run of the test. With timers or spin locks, it would cause unpredictable results when racing.
...
bash-5.1# ./test_progs -t lru_bug
test_progs-192 [000] d..21 354.838821: bpf_trace_printk: ref: 4
test_progs-192 [000] d..21 354.842824: bpf_trace_printk: ref: 5
bash-5.1# ./test_progs -t lru_bug
test_progs-193 [000] d..21 356.722813: bpf_trace_printk: ref: 5
test_progs-193 [000] d..21 356.727071: bpf_trace_printk: ref: 6... and so on.
--- .../selftests/bpf/prog_tests/lru_bug.c | 19 ++++++ tools/testing/selftests/bpf/progs/lru_bug.c | 67 +++++++++++++++++++ 2 files changed, 86 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/lru_bug.c create mode 100644 tools/testing/selftests/bpf/progs/lru_bug.c
tools/testing/selftests/bpf/prog_tests/lru_bug.c new file +19 -0
@@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <test_progs.h> + +#include "lru_bug.skel.h" + +void serial_test_lru_bug (void ) +{ +struct lru_bug *skel; +int ret; + + skel =lru_bug__open_and_load (); +if (!ASSERT_OK_PTR (skel,"lru_bug__open_and_load" )) +return ; + ret =lru_bug__attach (skel); +if (!ASSERT_OK (ret,"lru_bug__attach" )) +return ; +usleep (1 ); +ASSERT_OK (skel->data->result,"prealloc_lru_pop doesn't call check_and_init_map_value" ); +}
tools/testing/selftests/bpf/progs/lru_bug.c new file +67 -0
67 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH bpf v2 3/3] selftests/bpf: Add test for prealloc_lru_pop bug
Quoted 17 lines · Kumar Kartikeya Dwivedi
Add a regression test to check against invalid
check_and_init_map_valuecall insideprealloc_lru_pop.To actually observe a kind of problem this can cause, set debug to 1 when running the test locally without the fix. Then, just observe the refcount which keeps increasing on each run of the test. With timers or spin locks, it would cause unpredictable results when racing.
...
bash-5.1# ./test_progs -t
lru_bugtest_progs-192 [000] d..21 354.838821: bpf_trace_printk: ref: 4 test_progs-192 [000] d..21 354.842824: bpf_trace_printk: ref: 5 bash-5.1# ./test_progs -t lru_bug test_progs-193 [000] d..21 356.722813: bpf_trace_printk: ref: 5 test_progs-193 [000] d..21 356.727071: bpf_trace_printk: ref: 6
Quoted 3 lines
... and so on.
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
Ack with a minor nit below.
Quoted 19 lines
--- .../selftests/bpf/prog_tests/lru_bug.c | 19 ++++++ tools/testing/selftests/bpf/progs/lru_bug.c | 67 +++++++++++++++++++ 2 files changed, 86 insertions(+)create mode 100644
tools/testing/selftests/bpf/prog_tests/lru_bug.ccreate mode 100644tools/testing/selftests/bpf/progs/lru_bug.ctools/testing/selftests/bpf/prog_tests/lru_bug.c new file +6
-0@@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <test_progs.h> + +#include "lru_bug.skel.h" + +void serial_test_lru_bug (void )
tools/testing/selftests/bpf/prog_tests/lru_bug.c line 16
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/lru_bug.c new file +10
-0+{ +struct lru_bug *skel; +int ret; + + skel =lru_bug__open_and_load (); +if (!ASSERT_OK_PTR (skel,"lru_bug__open_and_load" )) +return ; + ret =lru_bug__attach (skel); +if (!ASSERT_OK (ret,"lru_bug__attach" )) +return ;
If not ASSERT_OK, should go to lru_bug__destroy(skel).
tools/testing/selftests/bpf/prog_tests/lru_bug.c line 18
Quoted 2 lines
tools/testing/selftests/bpf/prog_tests/lru_bug.c new file +2
-0+usleep (1 ); +ASSERT_OK (skel->data->result,"prealloc_lru_pop doesn't call check_and_init_map_value" );
Missing
lru_bug__destroy(skel);
.
Quoted 1 line
tools/testing/selftests/bpf/prog_tests/lru_bug.c new file +1
-0+}
[...]
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf v2 3/3] selftests/bpf: Add test for prealloc_lru_pop bug
Quoted 58 lines · Yonghong Song
Quoted 17 lines · Kumar Kartikeya Dwivedi
Add a regression test to check against invalid
check_and_init_map_valuecall insideprealloc_lru_pop.To actually observe a kind of problem this can cause, set debug to 1 when running the test locally without the fix. Then, just observe the refcount which keeps increasing on each run of the test. With timers or spin locks, it would cause unpredictable results when racing.
...
bash-5.1# ./test_progs -t
lru_bugtest_progs-192 [000] d..21 354.838821: bpf_trace_printk: ref: 4 test_progs-192 [000] d..21 354.842824: bpf_trace_printk: ref: 5 bash-5.1# ./test_progs -t lru_bug test_progs-193 [000] d..21 356.722813: bpf_trace_printk: ref: 5 test_progs-193 [000] d..21 356.727071: bpf_trace_printk: ref: 6Quoted 3 lines
... and so on.
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>Ack with a minor nit below.
Acked-by Yonghong Song <[email protected]>Quoted 19 lines
--- .../selftests/bpf/prog_tests/lru_bug.c | 19 ++++++ tools/testing/selftests/bpf/progs/lru_bug.c | 67 +++++++++++++++++++ 2 files changed, 86 insertions(+)create mode 100644
tools/testing/selftests/bpf/prog_tests/lru_bug.ccreate mode 100644tools/testing/selftests/bpf/progs/lru_bug.ctools/testing/selftests/bpf/prog_tests/lru_bug.c new file +6
-0@@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <test_progs.h> + +#include "lru_bug.skel.h" + +void serial_test_lru_bug (void )tools/testing/selftests/bpf/prog_tests/lru_bug.c line 16
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/lru_bug.c new file +10
-0+{ +struct lru_bug *skel; +int ret; + + skel =lru_bug__open_and_load (); +if (!ASSERT_OK_PTR (skel,"lru_bug__open_and_load" )) +return ; + ret =lru_bug__attach (skel); +if (!ASSERT_OK (ret,"lru_bug__attach" )) +return ;If not
ASSERT_OK, should go tolru_bug__destroy(skel).
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/lru_bug.c line 18
Quoted 2 lines
tools/testing/selftests/bpf/prog_tests/lru_bug.c new file +2
-0+usleep (1 ); +ASSERT_OK (skel->data->result,"prealloc_lru_pop doesn't call check_and_init_map_value" );Missing
lru_bug__destroy(skel);
Oops, thanks for catching, will fix.
Quoted 4 lines
.
Quoted 1 line
tools/testing/selftests/bpf/prog_tests/lru_bug.c new file +1
-0+}[...]
Quoted 11 lines · Kumar Kartikeya Dwivedi
Fix for a bug in
prelloc_lru_popspotted while reading the code, then a test + example that checks whether it is fixed.Changelog:
----------
v1 -> v2:
v1: https://lore.kernel.org/bpf/[email protected]* Expand commit log to include summary of the discussion with Yonghong
* Makelru_bugselftest serial to not mess up refcount formap_kptrtest
hmm. CI is still not happy.
Kumar Kartikeya Dwivedi
maintainerQuoted 11 lines · Alexei Starovoitov
Quoted 9 lines · Kumar Kartikeya Dwivedi
Fix for a bug in
prelloc_lru_popspotted while reading the code, then a test + example that checks whether it is fixed.Changelog:
----------
v1 -> v2:
v1: https://lore.kernel.org/bpf/[email protected]
Quoted 4 lines
Quoted 2 lines
* Expand commit log to include summary of the discussion with Yonghong
* Makelru_bugselftest serial to not mess up refcount formap_kptrtesthmm. CI is still not happy.
Yeah, I'll try to check for it differently.