Fix BPF struct_ops BTF cleanup race condition
cover Fix BPF struct_ops BTF cleanup race condition
This patch fixes a race condition in BPF struct_ops where consecutive test runs fail due to a timing mismatch between BTF object cleanup and BTF module removal.
When running BPF struct_ops tests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed from btf_modules list immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causing btf_try_get_module() to return NULL.
The fix synchronizes these cleanup operations by waiting for active BTF references to be released before completing module cleanup, eliminating the race condition.
This issue is reproducible with any of the following tests: global_map_resize, pro_epilogue, struct_ops_module and struct_ops_multi
./test_progs -t global_map_resize; ./test_progs -t global_map_resize
#134/1 global_map_resize/global_map_resize_bss:OK
#134/2 global_map_resize/global_map_resize_data:OK
#134/3 global_map_resize/global_map_resize_invalid:OK
#134 global_map_resize:OK
Summary: 1/3 PASSED, 0 SKIPPED, 0 FAILED global_map_resize_bss_subtest:PASS:test_global_map_resize__open 0 nsec
libbpf: map 'test_glo.bss': cannot be resized, last var must be an array
libbpf: map 'test_glo.bss': failed to adjust resized BTF, clearing BTF key/value info: -EINVAL global_map_resize_bss_subtest:PASS:bpf_map__set_value_size 0 nsec global_map_resize_bss_subtest:PASS:resize 0 nsec global_map_resize_bss_subtest:PASS:percpu_arr_resize 0 nsec global_map_resize_bss_subtest:PASS:array_len 0 nsec global_map_resize_bss_subtest:PASS:bpf_map__initial_value (ptr) 0 nsec global_map_resize_bss_subtest:PASS:bpf_map__initial_value (size) 0 nsec
libbpf: map 'st_ops_resize': failed to create: -EINVAL
libbpf: failed to load BPF skeleton 'test_global_map_resize': -EINVAL global_map_resize_bss_subtest:FAIL:test_global_map_resize__load unexpected error: -22 (errno 22)
#134/1 global_map_resize/global_map_resize_bss:FAIL
global_map_resize_data_subtest:PASS:test_global_map_resize__open 0 nsec global_map_resize_data_subtest:PASS:bpf_map__set_value_size 0 nsec global_map_resize_data_subtest:PASS:resize 0 nsec global_map_resize_data_subtest:PASS:percpu_arr_resize 0 nsec global_map_resize_data_subtest:PASS:array_len 0 nsec global_map_resize_data_subtest:PASS:bpf_map__initial_value (ptr) 0 nsec global_map_resize_data_subtest:PASS:bpf_map__initial_value (size) 0 nsec
libbpf: map 'st_ops_resize': failed to create: -EINVAL
libbpf: failed to load BPF skeleton 'test_global_map_resize': -EINVAL global_map_resize_data_subtest:FAIL:test_global_map_resize__load unexpected error: -22 (errno 22)
#134/2 global_map_resize/global_map_resize_data:FAIL
#134/3 global_map_resize/global_map_resize_invalid:OK
#134 global_map_resize:FAIL
After the fix, consecutive runs work reliably without requiring delays.
Gregory Bell (1):
bpf: Fix BTF module cleanup race condition in struct_ops
kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
Signature · 1 line
2.52.0
PATCH 1/1 bpf: Fix BTF module cleanup race condition in struct_ops
When running BPF struct_ops tests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed from btf_modules list immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causing btf_try_get_module() to return NULL.
Fix this by modifying MODULE_STATE_GOING handling to wait for active BTF references to be released before completing module cleanup. This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.
--- kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
kernel/bpf/btf.c +6 -1
@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; - +unsigned int timeout =1000 ; +while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }list_del (&btf_mod->list);if (btf_mod->sysfs_attr)sysfs_remove_bin_file (btf_kobj, btf_mod->sysfs_attr);
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH bpf-next 1/1] bpf: Fix BTF module cleanup race condition in struct_ops
This while loop calls msleep() while holding btf_module_mutex, which was acquired at the top of the MODULE_STATE_GOING case. This can hold the mutex for up to 1 second.
Quoted 5 lines
kernel/bpf/btf.c +0
-0
kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8391
Quoted 10 lines
kernel/bpf/btf.c +6
-0@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; +unsigned int timeout =1000 ; +while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }list_del (&btf_mod->list);
This while loop calls msleep() while holding btf_module_mutex, which was acquired at the top of the MODULE_STATE_GOING case. This can hold the mutex for up to 1 second.
During that time, all concurrent callers of btf_try_get_module(), btf_get_module_btf(), and any other module state notifications that need btf_module_mutex will block. This includes the BPF verifier path through __find_kfunc_desc_btf() and check_struct_ops_btf_id().
Additionally, module notifier callbacks run under the module subsystem's own locks. Sleeping here for up to a second delays all module load and unload operations system-wide.
Could the mutex be dropped before the wait, or could a different synchronization mechanism be used here, such as a completion?
Also, if the timeout expires with the refcount still above 1, the code falls through to list_del() and cleanup anyway. The commit message states:
Quoted 3 lines
This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.
This does not appear to be accurate, since the timeout makes the fix probabilistic rather than a guarantee. If a BTF reference is held for longer than 1 second, the original race condition would still occur.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/22923069980
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next 1/1] bpf: Fix BTF module cleanup race condition in struct_ops
In reply to Gregory BellQuoted 24 lines · Gregory Bell
When running BPF
struct_opstests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed frombtf_moduleslist immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causingbtf_try_get_module()to return NULL.Fix this by modifying
MODULE_STATE_GOINGhandling to wait for active BTF references to be released before completing module cleanup. This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.Signed-off-by Gregory Bell <[email protected]>--- kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/bpf/btf.c +0
-0
kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8390
Quoted 10 lines
kernel/bpf/btf.c +6
-1@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; - +unsigned int timeout =1000 ; +while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }
This isn't the right way to address such issues. Your diagnosis is correct but fix is just papering over real problems.
The better way would be to block the discovery of such BTFs once the module is unloaded.
I.e. once we return to userspace after module unload, subsequent calls to obtain an fd for such BTFs should not succeed.
If someone is racing to look it up while the unload happens, so be it, they will fail anyway.
I would use a bool flag in struct btf, set it inside the critical section at this point with WRITE_ONCE, then test it early in bpf_find_btf_id and bpf_core_find_cands with || in !btf_is_module condition.
Using READ_ONCE of course.
Also, we need to add it to btf_get_fd_by_id, so that once conversions from stale IDs to FDs don't succeed.
Looking at load_module_btfs(), it already handles the ENOENT case when id lookup succeeds, but fd lookup fails.
So for stale ids it should just skip over their errors if we return the same ENOENT as the refcount_inc_not_zero case.
We will still report stale ids, but not allow their conversion. We could filter id lookup too but it will penalize the common code and we need extra radix tree lookup just for BTF case.
For me locally, these changes seem to address the problem. I could send the fix, but if you would like to take a stab at the above change, please go ahead.
Quoted 7 lines
kernel/bpf/btf.c +0
-1list_del (&btf_mod->list);if (btf_mod->sysfs_attr)sysfs_remove_bin_file (btf_kobj, btf_mod->sysfs_attr); --2.52.0
Re: [PATCH bpf-next 1/1] bpf: Fix BTF module cleanup race condition in struct_ops
Quoted 31 lines · Kumar Kartikeya Dwivedi
Quoted 29 lines · Gregory Bell
When running BPF
struct_opstests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed frombtf_moduleslist immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causingbtf_try_get_module()to return NULL.Fix this by modifying
MODULE_STATE_GOINGhandling to wait for active BTF references to be released before completing module cleanup. This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.Signed-off-by Gregory Bell <[email protected]>--- kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/bpf/btf.c +1
-1@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; - +unsigned int timeout =1000 ;
Quoted 8 lines
kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8390
Quoted 5 lines
kernel/bpf/btf.c +5
-0+while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }This isn't the right way to address such issues. Your diagnosis is correct but fix is just papering over real problems.
Second that the fix is not right.
Quoted 23 lines
The better way would be to block the discovery of such BTFs once the module is unloaded.
I.e. once we return to userspace after module unload, subsequent calls to obtain an fd for such BTFs should not succeed.
If someone is racing to look it up while the unload happens, so be it, they will fail anyway.I would use a bool flag in struct btf, set it inside the critical section at this point with
WRITE_ONCE, then test it early inbpf_find_btf_idandbpf_core_find_candswith || in !btf_is_module condition.
UsingREAD_ONCEof course.
Also, we need to add it tobtf_get_fd_by_id, so that once conversions from stale IDs to FDs don't succeed.
Looking atload_module_btfs(), it already handles the ENOENT case when id lookup succeeds, but fd lookup fails.
So for stale ids it should just skip over their errors if we return the same ENOENT as therefcount_inc_not_zerocase.
We will still report stale ids, but not allow their conversion. We could filter id lookup too but it will penalize the common code and we need extra radix tree lookup just for BTF case.
Quoted 4 lines
For me locally, these changes seem to address the problem. I could send the fix, but if you would like to take a stab at the above change, please go ahead.
I guess this is worth a fix because it makes btf_get_fd_by_id() less likely to return to stale id. However, there is still a small window between btf_get_fd_by_id() and btf_try_get_module() so struct_ops map creation can still fail.
As for the solution, there already is btf_mod->flags. So maybe instead of adding a bool, just test the flag?
Quoted 10 lines
Quoted 9 lines
kernel/bpf/btf.c +0
-1list_del (&btf_mod->list);if (btf_mod->sysfs_attr)sysfs_remove_bin_file (btf_kobj, btf_mod->sysfs_attr); --2.52.0
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next 1/1] bpf: Fix BTF module cleanup race condition in struct_ops
Quoted 73 lines · Amery Hung
Quoted 31 lines · Kumar Kartikeya Dwivedi
Quoted 29 lines · Gregory Bell
When running BPF
struct_opstests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed frombtf_moduleslist immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causingbtf_try_get_module()to return NULL.Fix this by modifying
MODULE_STATE_GOINGhandling to wait for active BTF references to be released before completing module cleanup. This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.Signed-off-by Gregory Bell <[email protected]>--- kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/bpf/btf.c +1
-1@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; - +unsigned int timeout =1000 ;Quoted 8 lines
kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8390
Quoted 5 lines
kernel/bpf/btf.c +5
-0+while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }This isn't the right way to address such issues. Your diagnosis is correct but fix is just papering over real problems.
Second that the fix is not right.
Quoted 23 lines
The better way would be to block the discovery of such BTFs once the module is unloaded.
I.e. once we return to userspace after module unload, subsequent calls to obtain an fd for such BTFs should not succeed.
If someone is racing to look it up while the unload happens, so be it, they will fail anyway.I would use a bool flag in struct btf, set it inside the critical section at this point with
WRITE_ONCE, then test it early inbpf_find_btf_idandbpf_core_find_candswith || in !btf_is_module condition.
UsingREAD_ONCEof course.
Also, we need to add it tobtf_get_fd_by_id, so that once conversions from stale IDs to FDs don't succeed.
Looking atload_module_btfs(), it already handles the ENOENT case when id lookup succeeds, but fd lookup fails.
So for stale ids it should just skip over their errors if we return the same ENOENT as therefcount_inc_not_zerocase.
We will still report stale ids, but not allow their conversion. We could filter id lookup too but it will penalize the common code and we need extra radix tree lookup just for BTF case.Quoted 4 lines
For me locally, these changes seem to address the problem. I could send the fix, but if you would like to take a stab at the above change, please go ahead.
Quoted 4 lines
I guess this is worth a fix because it makes
btf_get_fd_by_id()less likely to return to stale id. However, there is still a small window betweenbtf_get_fd_by_id()andbtf_try_get_module()sostruct_opsmap creation can still fail.
That's fine, it means that module unload is happening in parallel, so it's racy and non-deterministic anyway.
When the user controls the module unload though, they should not be getting stale BTF fds once module unload has returned to user space. At that point all associated artifacts shouldn't be discoverable.
Quoted 3 lines
As for the solution, there already is
btf_mod->flags. So maybe instead of adding a bool, just test the flag?
You mean struct btf instead of btf_mod? I don't see the flags field there. The btf_mod->flags won't work, it's already deleted and freed by this time, and requires us to go first from the BTF to the btf_mod. So it has to be in struct btf.
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
kernel/bpf/btf.c +0
-1list_del (&btf_mod->list);if (btf_mod->sysfs_attr)sysfs_remove_bin_file (btf_kobj, btf_mod->sysfs_attr); --2.52.0
Re: [PATCH bpf-next 1/1] bpf: Fix BTF module cleanup race condition in struct_ops
Quoted 87 lines · Kumar Kartikeya Dwivedi
Quoted 73 lines · Amery Hung
Quoted 31 lines · Kumar Kartikeya Dwivedi
Quoted 29 lines · Gregory Bell
When running BPF
struct_opstests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed frombtf_moduleslist immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causingbtf_try_get_module()to return NULL.Fix this by modifying
MODULE_STATE_GOINGhandling to wait for active BTF references to be released before completing module cleanup. This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.Signed-off-by Gregory Bell <[email protected]>--- kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/bpf/btf.c +1
-1@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; - +unsigned int timeout =1000 ;Quoted 8 lines
kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8390
Quoted 5 lines
kernel/bpf/btf.c +5
-0+while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }This isn't the right way to address such issues. Your diagnosis is correct but fix is just papering over real problems.
Second that the fix is not right.
Quoted 23 lines
The better way would be to block the discovery of such BTFs once the module is unloaded.
I.e. once we return to userspace after module unload, subsequent calls to obtain an fd for such BTFs should not succeed.
If someone is racing to look it up while the unload happens, so be it, they will fail anyway.I would use a bool flag in struct btf, set it inside the critical section at this point with
WRITE_ONCE, then test it early inbpf_find_btf_idandbpf_core_find_candswith || in !btf_is_module condition.
UsingREAD_ONCEof course.
Also, we need to add it tobtf_get_fd_by_id, so that once conversions from stale IDs to FDs don't succeed.
Looking atload_module_btfs(), it already handles the ENOENT case when id lookup succeeds, but fd lookup fails.
So for stale ids it should just skip over their errors if we return the same ENOENT as therefcount_inc_not_zerocase.
We will still report stale ids, but not allow their conversion. We could filter id lookup too but it will penalize the common code and we need extra radix tree lookup just for BTF case.Quoted 4 lines
For me locally, these changes seem to address the problem. I could send the fix, but if you would like to take a stab at the above change, please go ahead.
Quoted 4 lines
I guess this is worth a fix because it makes
btf_get_fd_by_id()less likely to return to stale id. However, there is still a small window betweenbtf_get_fd_by_id()andbtf_try_get_module()sostruct_opsmap creation can still fail.That's fine, it means that module unload is happening in parallel, so it's racy and non-deterministic anyway.
When the user controls the module unload though, they should not be getting stale BTF fds once module unload has returned to user space. At that point all associated artifacts shouldn't be discoverable.Quoted 1 line
Quoted 8 lines
Quoted 2 lines
As for the solution, there already is
btf_mod->flags. So maybe instead of adding a bool, just test the flag?You mean struct btf instead of
btf_mod? I don't see the flags field there. Thebtf_mod->flagswon't work, it's already deleted and freed by this time, and requires us to go first from the BTF to thebtf_mod. So it has to be in struct btf.
I was thinking about updating btf_module->flags when handling MODULE_STATE_GOING. Then btf_get_fd_by_id() can call btf_try_get_module() to check flags if it's module BTF. If the module is already freed then it returns -ENOENT.
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 11 lines
kernel/bpf/btf.c +0
-1list_del (&btf_mod->list);if (btf_mod->sysfs_attr)sysfs_remove_bin_file (btf_kobj, btf_mod->sysfs_attr); --2.52.0
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next 1/1] bpf: Fix BTF module cleanup race condition in struct_ops
Quoted 99 lines · Amery Hung
Quoted 87 lines · Kumar Kartikeya Dwivedi
Quoted 73 lines · Amery Hung
Quoted 31 lines · Kumar Kartikeya Dwivedi
Quoted 29 lines · Gregory Bell
When running BPF
struct_opstests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed frombtf_moduleslist immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causingbtf_try_get_module()to return NULL.Fix this by modifying
MODULE_STATE_GOINGhandling to wait for active BTF references to be released before completing module cleanup. This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.Signed-off-by Gregory Bell <[email protected]>--- kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/bpf/btf.c +1
-1@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; - +unsigned int timeout =1000 ;Quoted 8 lines
kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8390
Quoted 5 lines
kernel/bpf/btf.c +5
-0+while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }This isn't the right way to address such issues. Your diagnosis is correct but fix is just papering over real problems.
Second that the fix is not right.
Quoted 23 lines
The better way would be to block the discovery of such BTFs once the module is unloaded.
I.e. once we return to userspace after module unload, subsequent calls to obtain an fd for such BTFs should not succeed.
If someone is racing to look it up while the unload happens, so be it, they will fail anyway.I would use a bool flag in struct btf, set it inside the critical section at this point with
WRITE_ONCE, then test it early inbpf_find_btf_idandbpf_core_find_candswith || in !btf_is_module condition.
UsingREAD_ONCEof course.
Also, we need to add it tobtf_get_fd_by_id, so that once conversions from stale IDs to FDs don't succeed.
Looking atload_module_btfs(), it already handles the ENOENT case when id lookup succeeds, but fd lookup fails.
So for stale ids it should just skip over their errors if we return the same ENOENT as therefcount_inc_not_zerocase.
We will still report stale ids, but not allow their conversion. We could filter id lookup too but it will penalize the common code and we need extra radix tree lookup just for BTF case.Quoted 4 lines
For me locally, these changes seem to address the problem. I could send the fix, but if you would like to take a stab at the above change, please go ahead.
Quoted 4 lines
I guess this is worth a fix because it makes
btf_get_fd_by_id()less likely to return to stale id. However, there is still a small window betweenbtf_get_fd_by_id()andbtf_try_get_module()sostruct_opsmap creation can still fail.That's fine, it means that module unload is happening in parallel, so it's racy and non-deterministic anyway.
When the user controls the module unload though, they should not be getting stale BTF fds once module unload has returned to user space. At that point all associated artifacts shouldn't be discoverable.Quoted 1 line
Quoted 8 lines
Quoted 2 lines
As for the solution, there already is
btf_mod->flags. So maybe instead of adding a bool, just test the flag?You mean struct btf instead of
btf_mod? I don't see the flags field there. Thebtf_mod->flagswon't work, it's already deleted and freed by this time, and requires us to go first from the BTF to thebtf_mod. So it has to be in struct btf.
Quoted 4 lines
I was thinking about updating
btf_module->flagswhen handlingMODULE_STATE_GOING. Thenbtf_get_fd_by_id()can callbtf_try_get_module()to check flags if it's module BTF. If the module is already freed then it returns-ENOENT.
Hmm, interesting thought.
I guess the only downside is that it's O(n) lookup under mutex, which can be costly with lots of modules, and always penalizes get_fd_by_id() for handling race.
We also need to repeat it in bpf_core_find_cands and bpf_find_btf_id, where we may repeat it multiple times in the loop, so the cost is amplified.
Let's see what other maintainers think.
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
kernel/bpf/btf.c +0
-1list_del (&btf_mod->list);if (btf_mod->sysfs_attr)sysfs_remove_bin_file (btf_kobj, btf_mod->sysfs_attr); --2.52.0
Martin KaFai Lau
reviewerRe: [PATCH bpf-next 1/1] bpf: Fix BTF module cleanup race condition in struct_ops
Quoted 49 lines · Kumar Kartikeya Dwivedi
Quoted 48 lines · Amery Hung
Quoted 45 lines · Kumar Kartikeya Dwivedi
Quoted 43 lines · Amery Hung
Quoted 39 lines · Kumar Kartikeya Dwivedi
Quoted 24 lines · Gregory Bell
When running BPF
struct_opstests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed frombtf_moduleslist immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causingbtf_try_get_module()to return NULL.Fix this by modifying
MODULE_STATE_GOINGhandling to wait for active BTF references to be released before completing module cleanup. This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.Signed-off-by Gregory Bell <[email protected]>--- kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/bpf/btf.c +0
-0kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8390
Quoted 10 lines
kernel/bpf/btf.c +6
-1@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; - +unsigned int timeout =1000 ; +while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }This isn't the right way to address such issues. Your diagnosis is correct but fix is just papering over real problems.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Second that the fix is not right.
pw-bot: cr
Quoted 55 lines
Quoted 50 lines
Quoted 41 lines
Quoted 29 lines
Quoted 24 lines
The better way would be to block the discovery of such BTFs once the module is unloaded.
I.e. once we return to userspace after module unload, subsequent calls to obtain an fd for such BTFs should not succeed.
If someone is racing to look it up while the unload happens, so be it, they will fail anyway.I would use a bool flag in struct btf, set it inside the critical section at this point with
WRITE_ONCE, then test it early inbpf_find_btf_idandbpf_core_find_candswith || in !btf_is_module condition.
UsingREAD_ONCEof course.
Also, we need to add it tobtf_get_fd_by_id, so that once conversions from stale IDs to FDs don't succeed.
Looking atload_module_btfs(), it already handles the ENOENT case when id lookup succeeds, but fd lookup fails.
So for stale ids it should just skip over their errors if we return the same ENOENT as therefcount_inc_not_zerocase.
We will still report stale ids, but not allow their conversion. We could filter id lookup too but it will penalize the common code and we need extra radix tree lookup just for BTF case.Quoted 4 lines
For me locally, these changes seem to address the problem. I could send the fix, but if you would like to take a stab at the above change, please go ahead.
Quoted 4 lines
I guess this is worth a fix because it makes
btf_get_fd_by_id()less likely to return to stale id. However, there is still a small window betweenbtf_get_fd_by_id()andbtf_try_get_module()sostruct_opsmap creation can still fail.That's fine, it means that module unload is happening in parallel, so it's racy and non-deterministic anyway.
When the user controls the module unload though, they should not be getting stale BTF fds once module unload has returned to user space. At that point all associated artifacts shouldn't be discoverable.Quoted 1 line
Quoted 8 lines
Quoted 2 lines
As for the solution, there already is
btf_mod->flags. So maybe instead of adding a bool, just test the flag?You mean struct btf instead of
btf_mod? I don't see the flags field there. Thebtf_mod->flagswon't work, it's already deleted and freed by this time, and requires us to go first from the BTF to thebtf_mod. So it has to be in struct btf.Quoted 4 lines
I was thinking about updating
btf_module->flagswhen handlingMODULE_STATE_GOING. Thenbtf_get_fd_by_id()can callbtf_try_get_module()to check flags if it's module BTF. If the module is already freed then it returns-ENOENT.
Quoted 8 lines
Hmm, interesting thought.
I guess the only downside is that it's O(n) lookup under mutex, which can be costly with lots of modules, and always penalizesget_fd_by_id()for handling race.
We also need to repeat it inbpf_core_find_candsandbpf_find_btf_id, where we may repeat it multiple times in the loop, so the cost is amplified.
Is btf->id still needed somewhere after MODULE_STATE_GOING? Otherwise, directly removing btf->id from btf_idr in btf_module_notify() should stop the discovery.
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next 1/1] bpf: Fix BTF module cleanup race condition in struct_ops
Quoted 120 lines · Martin KaFai Lau
Quoted 49 lines · Kumar Kartikeya Dwivedi
Quoted 48 lines · Amery Hung
Quoted 45 lines · Kumar Kartikeya Dwivedi
Quoted 43 lines · Amery Hung
Quoted 39 lines · Kumar Kartikeya Dwivedi
Quoted 24 lines · Gregory Bell
When running BPF
struct_opstests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed frombtf_moduleslist immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causingbtf_try_get_module()to return NULL.Fix this by modifying
MODULE_STATE_GOINGhandling to wait for active BTF references to be released before completing module cleanup. This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.Signed-off-by Gregory Bell <[email protected]>--- kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/bpf/btf.c +0
-0kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8390
Quoted 10 lines
kernel/bpf/btf.c +6
-1@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; - +unsigned int timeout =1000 ; +while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }This isn't the right way to address such issues. Your diagnosis is correct but fix is just papering over real problems.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Second that the fix is not right.
pw-bot: cr
Quoted 55 lines
Quoted 50 lines
Quoted 41 lines
Quoted 29 lines
Quoted 24 lines
The better way would be to block the discovery of such BTFs once the module is unloaded.
I.e. once we return to userspace after module unload, subsequent calls to obtain an fd for such BTFs should not succeed.
If someone is racing to look it up while the unload happens, so be it, they will fail anyway.I would use a bool flag in struct btf, set it inside the critical section at this point with
WRITE_ONCE, then test it early inbpf_find_btf_idandbpf_core_find_candswith || in !btf_is_module condition.
UsingREAD_ONCEof course.
Also, we need to add it tobtf_get_fd_by_id, so that once conversions from stale IDs to FDs don't succeed.
Looking atload_module_btfs(), it already handles the ENOENT case when id lookup succeeds, but fd lookup fails.
So for stale ids it should just skip over their errors if we return the same ENOENT as therefcount_inc_not_zerocase.
We will still report stale ids, but not allow their conversion. We could filter id lookup too but it will penalize the common code and we need extra radix tree lookup just for BTF case.Quoted 4 lines
For me locally, these changes seem to address the problem. I could send the fix, but if you would like to take a stab at the above change, please go ahead.
Quoted 4 lines
I guess this is worth a fix because it makes
btf_get_fd_by_id()less likely to return to stale id. However, there is still a small window betweenbtf_get_fd_by_id()andbtf_try_get_module()sostruct_opsmap creation can still fail.That's fine, it means that module unload is happening in parallel, so it's racy and non-deterministic anyway.
When the user controls the module unload though, they should not be getting stale BTF fds once module unload has returned to user space. At that point all associated artifacts shouldn't be discoverable.Quoted 1 line
Quoted 8 lines
Quoted 2 lines
As for the solution, there already is
btf_mod->flags. So maybe instead of adding a bool, just test the flag?You mean struct btf instead of
btf_mod? I don't see the flags field there. Thebtf_mod->flagswon't work, it's already deleted and freed by this time, and requires us to go first from the BTF to thebtf_mod. So it has to be in struct btf.Quoted 4 lines
I was thinking about updating
btf_module->flagswhen handlingMODULE_STATE_GOING. Thenbtf_get_fd_by_id()can callbtf_try_get_module()to check flags if it's module BTF. If the module is already freed then it returns-ENOENT.Quoted 8 lines
Hmm, interesting thought.
I guess the only downside is that it's O(n) lookup under mutex, which can be costly with lots of modules, and always penalizesget_fd_by_id()for handling race.
We also need to repeat it inbpf_core_find_candsandbpf_find_btf_id, where we may repeat it multiple times in the loop, so the cost is amplified.
Quoted 3 lines
Is
btf->idstill needed somewhere afterMODULE_STATE_GOING? Otherwise, directly removingbtf->idfrombtf_idrinbtf_module_notify()should stop the discovery.
I looked at this, I think we will need to reset the btf->id too. Otherwise while btf_free_id is idempotent (no-op on free id), the released id can be reused and if we don't guard btf_free_id in btf_put it will remove it for some other BTF initialized in the meantime. We can likely set a flag or something to skip it in btf_put(). As long as the module is holding a reference nothing else should do it.
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next 1/1] bpf: Fix BTF module cleanup race condition in struct_ops
Quoted 126 lines · Kumar Kartikeya Dwivedi
Quoted 120 lines · Martin KaFai Lau
Quoted 49 lines · Kumar Kartikeya Dwivedi
Quoted 48 lines · Amery Hung
Quoted 45 lines · Kumar Kartikeya Dwivedi
Quoted 43 lines · Amery Hung
Quoted 39 lines · Kumar Kartikeya Dwivedi
Quoted 24 lines · Gregory Bell
When running BPF
struct_opstests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed frombtf_moduleslist immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causingbtf_try_get_module()to return NULL.Fix this by modifying
MODULE_STATE_GOINGhandling to wait for active BTF references to be released before completing module cleanup. This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.Signed-off-by Gregory Bell <[email protected]>--- kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/bpf/btf.c +0
-0kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8390
Quoted 10 lines
kernel/bpf/btf.c +6
-1@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; - +unsigned int timeout =1000 ; +while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }This isn't the right way to address such issues. Your diagnosis is correct but fix is just papering over real problems.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Second that the fix is not right.
pw-bot: cr
Quoted 55 lines
Quoted 50 lines
Quoted 41 lines
Quoted 29 lines
Quoted 24 lines
The better way would be to block the discovery of such BTFs once the module is unloaded.
I.e. once we return to userspace after module unload, subsequent calls to obtain an fd for such BTFs should not succeed.
If someone is racing to look it up while the unload happens, so be it, they will fail anyway.I would use a bool flag in struct btf, set it inside the critical section at this point with
WRITE_ONCE, then test it early inbpf_find_btf_idandbpf_core_find_candswith || in !btf_is_module condition.
UsingREAD_ONCEof course.
Also, we need to add it tobtf_get_fd_by_id, so that once conversions from stale IDs to FDs don't succeed.
Looking atload_module_btfs(), it already handles the ENOENT case when id lookup succeeds, but fd lookup fails.
So for stale ids it should just skip over their errors if we return the same ENOENT as therefcount_inc_not_zerocase.
We will still report stale ids, but not allow their conversion. We could filter id lookup too but it will penalize the common code and we need extra radix tree lookup just for BTF case.Quoted 4 lines
For me locally, these changes seem to address the problem. I could send the fix, but if you would like to take a stab at the above change, please go ahead.
Quoted 4 lines
I guess this is worth a fix because it makes
btf_get_fd_by_id()less likely to return to stale id. However, there is still a small window betweenbtf_get_fd_by_id()andbtf_try_get_module()sostruct_opsmap creation can still fail.That's fine, it means that module unload is happening in parallel, so it's racy and non-deterministic anyway.
When the user controls the module unload though, they should not be getting stale BTF fds once module unload has returned to user space. At that point all associated artifacts shouldn't be discoverable.Quoted 1 line
Quoted 8 lines
Quoted 2 lines
As for the solution, there already is
btf_mod->flags. So maybe instead of adding a bool, just test the flag?You mean struct btf instead of
btf_mod? I don't see the flags field there. Thebtf_mod->flagswon't work, it's already deleted and freed by this time, and requires us to go first from the BTF to thebtf_mod. So it has to be in struct btf.Quoted 4 lines
I was thinking about updating
btf_module->flagswhen handlingMODULE_STATE_GOING. Thenbtf_get_fd_by_id()can callbtf_try_get_module()to check flags if it's module BTF. If the module is already freed then it returns-ENOENT.Quoted 8 lines
Hmm, interesting thought.
I guess the only downside is that it's O(n) lookup under mutex, which can be costly with lots of modules, and always penalizesget_fd_by_id()for handling race.
We also need to repeat it inbpf_core_find_candsandbpf_find_btf_id, where we may repeat it multiple times in the loop, so the cost is amplified.Quoted 3 lines
Is
btf->idstill needed somewhere afterMODULE_STATE_GOING? Otherwise, directly removingbtf->idfrombtf_idrinbtf_module_notify()should stop the discovery.
Quoted 6 lines
I looked at this, I think we will need to reset the
btf->idtoo. Otherwise whilebtf_free_idis idempotent (no-op on free id), the released id can be reused and if we don't guardbtf_free_idinbtf_putit will remove it for some other BTF initialized in the meantime. We can likely set a flag or something to skip it inbtf_put(). As long as the module is holding a reference nothing else should do it.
Another way to make the check is skip btf_free_id() in btf_put() for btf_is_module(), and always do it in the MODULE_STATE_GOING case. The module is always going to release its reference, and precisely at that point we want the idr to be freed.
Re: [PATCH bpf-next 1/1] bpf: Fix BTF module cleanup race condition in struct_ops
In reply to Kumar Kartikeya DwivediQuoted 125 lines · Kumar Kartikeya Dwivedi
Quoted 120 lines · Martin KaFai Lau
Quoted 49 lines · Kumar Kartikeya Dwivedi
Quoted 48 lines · Amery Hung
Quoted 45 lines · Kumar Kartikeya Dwivedi
Quoted 43 lines · Amery Hung
Quoted 39 lines · Kumar Kartikeya Dwivedi
Quoted 24 lines · Gregory Bell
When running BPF
struct_opstests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed frombtf_moduleslist immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causingbtf_try_get_module()to return NULL.Fix this by modifying
MODULE_STATE_GOINGhandling to wait for active BTF references to be released before completing module cleanup. This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.Signed-off-by Gregory Bell <[email protected]>--- kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/bpf/btf.c +0
-0kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8390
Quoted 10 lines
kernel/bpf/btf.c +6
-1@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; - +unsigned int timeout =1000 ; +while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }This isn't the right way to address such issues. Your diagnosis is correct but fix is just papering over real problems.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Second that the fix is not right.
pw-bot: cr
Quoted 55 lines
Quoted 50 lines
Quoted 41 lines
Quoted 29 lines
Quoted 24 lines
The better way would be to block the discovery of such BTFs once the module is unloaded.
I.e. once we return to userspace after module unload, subsequent calls to obtain an fd for such BTFs should not succeed.
If someone is racing to look it up while the unload happens, so be it, they will fail anyway.I would use a bool flag in struct btf, set it inside the critical section at this point with
WRITE_ONCE, then test it early inbpf_find_btf_idandbpf_core_find_candswith || in !btf_is_module condition.
UsingREAD_ONCEof course.
Also, we need to add it tobtf_get_fd_by_id, so that once conversions from stale IDs to FDs don't succeed.
Looking atload_module_btfs(), it already handles the ENOENT case when id lookup succeeds, but fd lookup fails.
So for stale ids it should just skip over their errors if we return the same ENOENT as therefcount_inc_not_zerocase.
We will still report stale ids, but not allow their conversion. We could filter id lookup too but it will penalize the common code and we need extra radix tree lookup just for BTF case.Quoted 4 lines
For me locally, these changes seem to address the problem. I could send the fix, but if you would like to take a stab at the above change, please go ahead.
Quoted 4 lines
I guess this is worth a fix because it makes
btf_get_fd_by_id()less likely to return to stale id. However, there is still a small window betweenbtf_get_fd_by_id()andbtf_try_get_module()sostruct_opsmap creation can still fail.That's fine, it means that module unload is happening in parallel, so it's racy and non-deterministic anyway.
When the user controls the module unload though, they should not be getting stale BTF fds once module unload has returned to user space. At that point all associated artifacts shouldn't be discoverable.Quoted 1 line
Quoted 8 lines
Quoted 2 lines
As for the solution, there already is
btf_mod->flags. So maybe instead of adding a bool, just test the flag?You mean struct btf instead of
btf_mod? I don't see the flags field there. Thebtf_mod->flagswon't work, it's already deleted and freed by this time, and requires us to go first from the BTF to thebtf_mod. So it has to be in struct btf.Quoted 4 lines
I was thinking about updating
btf_module->flagswhen handlingMODULE_STATE_GOING. Thenbtf_get_fd_by_id()can callbtf_try_get_module()to check flags if it's module BTF. If the module is already freed then it returns-ENOENT.Quoted 8 lines
Hmm, interesting thought.
I guess the only downside is that it's O(n) lookup under mutex, which can be costly with lots of modules, and always penalizesget_fd_by_id()for handling race.
We also need to repeat it inbpf_core_find_candsandbpf_find_btf_id, where we may repeat it multiple times in the loop, so the cost is amplified.Quoted 3 lines
Is
btf->idstill needed somewhere afterMODULE_STATE_GOING? Otherwise, directly removingbtf->idfrombtf_idrinbtf_module_notify()should stop the discovery.
Quoted 7 lines
I looked at this, I think we will need to reset the
btf->idtoo. Otherwise whilebtf_free_idis idempotent (no-op on free id), the released id can be reused and if we don't guardbtf_free_idinbtf_putit will remove it for some other BTF initialized in the meantime. We can likely set a flag or something to skip it inbtf_put(). As long as the module is holding a reference nothing else should do it.
I wonder if it'd make sense to reduce the cost of btf -> module lookup so that we could check the flag in contexts like btf_get_fd_by_id()? AI suggested reworking btf_module lookup from a linked list to a pair of xarrays keyed by struct module * and struct btf *, and it seems reasonable (testing it now). An easy win might be to get rid of the strcmp() to vmlinux in btf_is_module(), replacing it with the simpler predicate
btf->kernel_btf && btf->base_btf;
to reduce overhead of lots of btf_is_module() lookups. What do you think?
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next 1/1] bpf: Fix BTF module cleanup race condition in struct_ops
Quoted 142 lines · Alan Maguire
Quoted 125 lines · Kumar Kartikeya Dwivedi
Quoted 120 lines · Martin KaFai Lau
Quoted 49 lines · Kumar Kartikeya Dwivedi
Quoted 48 lines · Amery Hung
Quoted 45 lines · Kumar Kartikeya Dwivedi
Quoted 43 lines · Amery Hung
Quoted 39 lines · Kumar Kartikeya Dwivedi
Quoted 24 lines · Gregory Bell
When running BPF
struct_opstests consecutively, the second test fails because BTF object references take longer to be released while their associated module is removed frombtf_moduleslist immediately during module unload. This creates a race condition where the second test retrieves the cached BTF object but cannot find valid module information, causingbtf_try_get_module()to return NULL.Fix this by modifying
MODULE_STATE_GOINGhandling to wait for active BTF references to be released before completing module cleanup. This ensures BTF objects are properly cleaned up from the cache before their module metadata is removed, eliminating the timing window that caused the race condition.Signed-off-by Gregory Bell <[email protected]>--- kernel/bpf/btf.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)kernel/bpf/btf.c +0
-0kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8390
Quoted 10 lines
kernel/bpf/btf.c +6
-1@@ -8382,7 +8382,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,list_for_each_entry_safe (btf_mod, tmp, &btf_modules, list) {if (btf_mod->module != module)continue ; - +unsigned int timeout =1000 ; +while (refcount_read (&btf_mod->btf->refcnt) >1 + && timeout >0 ) { +msleep (1 ); + timeout--; + }This isn't the right way to address such issues. Your diagnosis is correct but fix is just papering over real problems.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Second that the fix is not right.
pw-bot: cr
Quoted 55 lines
Quoted 50 lines
Quoted 41 lines
Quoted 29 lines
Quoted 24 lines
The better way would be to block the discovery of such BTFs once the module is unloaded.
I.e. once we return to userspace after module unload, subsequent calls to obtain an fd for such BTFs should not succeed.
If someone is racing to look it up while the unload happens, so be it, they will fail anyway.I would use a bool flag in struct btf, set it inside the critical section at this point with
WRITE_ONCE, then test it early inbpf_find_btf_idandbpf_core_find_candswith || in !btf_is_module condition.
UsingREAD_ONCEof course.
Also, we need to add it tobtf_get_fd_by_id, so that once conversions from stale IDs to FDs don't succeed.
Looking atload_module_btfs(), it already handles the ENOENT case when id lookup succeeds, but fd lookup fails.
So for stale ids it should just skip over their errors if we return the same ENOENT as therefcount_inc_not_zerocase.
We will still report stale ids, but not allow their conversion. We could filter id lookup too but it will penalize the common code and we need extra radix tree lookup just for BTF case.Quoted 4 lines
For me locally, these changes seem to address the problem. I could send the fix, but if you would like to take a stab at the above change, please go ahead.
Quoted 4 lines
I guess this is worth a fix because it makes
btf_get_fd_by_id()less likely to return to stale id. However, there is still a small window betweenbtf_get_fd_by_id()andbtf_try_get_module()sostruct_opsmap creation can still fail.That's fine, it means that module unload is happening in parallel, so it's racy and non-deterministic anyway.
When the user controls the module unload though, they should not be getting stale BTF fds once module unload has returned to user space. At that point all associated artifacts shouldn't be discoverable.Quoted 1 line
Quoted 8 lines
Quoted 2 lines
As for the solution, there already is
btf_mod->flags. So maybe instead of adding a bool, just test the flag?You mean struct btf instead of
btf_mod? I don't see the flags field there. Thebtf_mod->flagswon't work, it's already deleted and freed by this time, and requires us to go first from the BTF to thebtf_mod. So it has to be in struct btf.Quoted 4 lines
I was thinking about updating
btf_module->flagswhen handlingMODULE_STATE_GOING. Thenbtf_get_fd_by_id()can callbtf_try_get_module()to check flags if it's module BTF. If the module is already freed then it returns-ENOENT.Quoted 8 lines
Hmm, interesting thought.
I guess the only downside is that it's O(n) lookup under mutex, which can be costly with lots of modules, and always penalizesget_fd_by_id()for handling race.
We also need to repeat it inbpf_core_find_candsandbpf_find_btf_id, where we may repeat it multiple times in the loop, so the cost is amplified.Quoted 3 lines
Is
btf->idstill needed somewhere afterMODULE_STATE_GOING? Otherwise, directly removingbtf->idfrombtf_idrinbtf_module_notify()should stop the discovery.Quoted 7 lines
I looked at this, I think we will need to reset the
btf->idtoo. Otherwise whilebtf_free_idis idempotent (no-op on free id), the released id can be reused and if we don't guardbtf_free_idinbtf_putit will remove it for some other BTF initialized in the meantime. We can likely set a flag or something to skip it inbtf_put(). As long as the module is holding a reference nothing else should do it.I wonder if it'd make sense to reduce the cost of btf -> module lookup so that we could check the flag in contexts like
btf_get_fd_by_id()? AI suggested reworkingbtf_modulelookup from a linked list to a pair of xarrays keyed by struct module * and struct btf *, and it seems reasonable (testing it now). An easy win might be to get rid of thestrcmp()to vmlinux inbtf_is_module(), replacing it with the simpler predicate
Quoted 3 lines
btf->kernel_btf && btf->base_btf;to reduce overhead of lots of
btf_is_module()lookups. What do you think?
That is probably worth doing if btf_module lookup is taking up significant CPU time, but I think as a fix it would be too much churn, and be harder to backport.
I shared a diff trying out Martin's idea and it seems to fix the issue.
Kumar Kartikeya Dwivedi
maintainerPATCH 1/1 bpf: Release module BTF IDR before module unload
In reply to Martin KaFai Lau--- kernel/bpf/btf.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
kernel/bpf/btf.c +14 -1
@@ -1883,10 +1883,17 @@ void btf_get(struct btf *btf)refcount_inc (&btf->refcnt); } +static bool btf_is_module (const struct btf *btf); +void btf_put (struct btf *btf) {if (btf &&refcount_dec_and_test (&btf->refcnt)) { -btf_free_id (btf); +/* +* IDR for module BTFs is freed when module's BTF reference is +* dropped, which may happen before the final put. +*/ +if (!btf_is_module (btf)) +btf_free_id (btf);call_rcu (&btf->rcu, btf_free_rcu); } }
@@ -8383,6 +8390,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,if (btf_mod->module != module)continue ; +/* +* For modules, we do the freeing of BTF IDR as soon as +* module goes away to disable BTF discovery, since the +* btf_try_get_module() on such BTFs will fail. +*/ +btf_free_id (btf_mod->btf);list_del (&btf_mod->list);if (btf_mod->sysfs_attr)sysfs_remove_bin_file (btf_kobj, btf_mod->sysfs_attr);
Signature · 1 line
2.52.0
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH] bpf: Release module BTF IDR before module unload
Quoted 11 lines · Kumar Kartikeya Dwivedi
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- kernel/bpf/btf.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)kernel/bpf/btf.c +0
-0
kernel/bpf/btf.c void btf_get(struct btf *btf) · line 1887
Quoted 5 lines
kernel/bpf/btf.c +2
-0@@ -1883,10 +1883,17 @@ void btf_get(struct btf *btf)refcount_inc (&btf->refcnt); } +static bool btf_is_module (const struct btf *btf); +
This probably needs to be 'bool btf_is_module(const struct btf *btf);' , i.e. without static.
I didn't get a warning in my build, strangely enough, but nevertheless.
Gregory, please give it a spin and see if it fixes the bug for you. Martin, please let me know if this looks ok to you.
Quoted 29 lines
kernel/bpf/btf.c +12
-2void btf_put (struct btf *btf) {if (btf &&refcount_dec_and_test (&btf->refcnt)) { -btf_free_id (btf); +/* +* IDR for module BTFs is freed when module's BTF reference is +* dropped, which may happen before the final put. +*/ +if (!btf_is_module (btf)) +btf_free_id (btf);call_rcu (&btf->rcu, btf_free_rcu); } }@@ -8383,6 +8390,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,if (btf_mod->module != module)continue ; +/* +* For modules, we do the freeing of BTF IDR as soon as +* module goes away to disable BTF discovery, since the +* btf_try_get_module() on such BTFs will fail. +*/ +btf_free_id (btf_mod->btf);list_del (&btf_mod->list);if (btf_mod->sysfs_attr)sysfs_remove_bin_file (btf_kobj, btf_mod->sysfs_attr); --2.52.0
Re: [PATCH] bpf: Release module BTF IDR before module unload
Thanks for this Kumar! I confirmed that this does fix the bug for us.
Quoted 6 lines
kernel/bpf/btf.c void btf_get(struct btf *btf) · line 1887
Quoted 2 lines
kernel/bpf/btf.c +2
-0+static bool btf_is_module (const struct btf *btf); +This probably needs to be 'bool
btf_is_module(const struct btf *btf);' , i.e. without static.
I didn't get a warning in my build, strangely enough, but nevertheless.
For what its worth for me it did throw an error for the static declaration and I had to drop the static keyword:
kernel/bpf/btf.c:1886:13: error: static declaration of ‘btf_is_module’ follows non-static declaration
1886 | static bool btf_is_module(const struct btf *btf);
| ^~~~~~~~~~~~~
In file included from ./include/linux/bpf.h:29,
from ./include/linux/bpf_verifier.h:7,
from kernel/bpf/btf.c:19:
./include/linux/btf.h:219:6: note: previous declaration of
‘btf_is_module’ with type ‘bool(const struct btf *)’ {aka ‘_Bool(co
nst struct btf *)’}
219 | bool btf_is_module(const struct btf *btf);
| ^~~~~~~~~~~~~make[4]: *** [scripts/Makefile.build:289: kernel/bpf/btf.o] Error 1
Quoted 56 lines · Kumar Kartikeya Dwivedi
Quoted 11 lines · Kumar Kartikeya Dwivedi
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- kernel/bpf/btf.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)kernel/bpf/btf.c +0
-0kernel/bpf/btf.c void btf_get(struct btf *btf) · line 1887
Quoted 5 lines
kernel/bpf/btf.c +2
-0@@ -1883,10 +1883,17 @@ void btf_get(struct btf *btf)refcount_inc (&btf->refcnt); } +static bool btf_is_module (const struct btf *btf); +This probably needs to be 'bool
btf_is_module(const struct btf *btf);' , i.e. without static.
I didn't get a warning in my build, strangely enough, but nevertheless.Gregory, please give it a spin and see if it fixes the bug for you. Martin, please let me know if this looks ok to you.
Quoted 29 lines
kernel/bpf/btf.c +12
-2void btf_put (struct btf *btf) {if (btf &&refcount_dec_and_test (&btf->refcnt)) { -btf_free_id (btf); +/* +* IDR for module BTFs is freed when module's BTF reference is +* dropped, which may happen before the final put. +*/ +if (!btf_is_module (btf)) +btf_free_id (btf);call_rcu (&btf->rcu, btf_free_rcu); } }@@ -8383,6 +8390,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,if (btf_mod->module != module)continue ; +/* +* For modules, we do the freeing of BTF IDR as soon as +* module goes away to disable BTF discovery, since the +* btf_try_get_module() on such BTFs will fail. +*/ +btf_free_id (btf_mod->btf);list_del (&btf_mod->list);if (btf_mod->sysfs_attr)sysfs_remove_bin_file (btf_kobj, btf_mod->sysfs_attr); --2.52.0
[email protected]
AI reviewRe: [PATCH] bpf: Release module BTF IDR before module unload
In reply to Kumar Kartikeya Dwivedibtf_is_module() is already declared with external linkage in include/linux/btf.h:
kernel/bpf/btf.c void btf_get(struct btf *btf) · line 1886
Quoted 8 lines
kernel/bpf/btf.c +1
-0@@ -1883,10 +1883,17 @@ void btf_get(struct btf *btf)refcount_inc (&btf->refcnt); } +static bool btf_is_module (const struct btf *btf);
btf_is_module() is already declared with external linkage in include/linux/btf.h:
bool btf_is_module(const struct btf *btf);
and btf.c includes that header. Won't the static forward declaration here cause a build error? GCC rejects a static declaration that follows a non-static declaration of the same function. Dropping the 'static' keyword from this forward declaration should fix it.
kernel/bpf/btf.c void btf_get(struct btf *btf) · line 1888
Quoted 2 lines
kernel/bpf/btf.c +1
-0+void btf_put (struct btf *btf)
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/22954450342
Martin KaFai Lau
reviewerRe: [PATCH] bpf: Release module BTF IDR before module unload
In reply to Kumar Kartikeya DwivediQuoted 29 lines · Kumar Kartikeya Dwivedi
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- kernel/bpf/btf.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)kernel/bpf/btf.c +8
-1@@ -1883,10 +1883,17 @@ void btf_get(struct btf *btf)refcount_inc (&btf->refcnt); } +static bool btf_is_module (const struct btf *btf); +void btf_put (struct btf *btf) {if (btf &&refcount_dec_and_test (&btf->refcnt)) { -btf_free_id (btf); +/* +* IDR for module BTFs is freed when module's BTF reference is +* dropped, which may happen before the final put. +*/ +if (!btf_is_module (btf)) +btf_free_id (btf);call_rcu (&btf->rcu, btf_free_rcu); } }
kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8399
Quoted 10 lines
kernel/bpf/btf.c +6
-0@@ -8383,6 +8390,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,if (btf_mod->module != module)continue ; +/* +* For modules, we do the freeing of BTF IDR as soon as +* module goes away to disable BTF discovery, since the +* btf_try_get_module() on such BTFs will fail. +*/ +btf_free_id (btf_mod->btf);list_del (&btf_mod->list);
The change makes sense.
I looked up a bit because I am interested in who will use the module's btf->id without holding the module's refcount. fwiw, here is what I read.
For non struct_ops prog, the module refcount is held in prog->aux->mod after commit 31bf1dbccfb0, so should be fine on the bpf_trampoline_compute_key() and bpf_prog_get_info_by_fd().
For struct_ops prog, the module refcount is not held in prog->aux->mod. I don't think it has to since it is not directly attached to the module. The only downside is the bpf_prog_get_info_by_fd() may report an out-dated info.attach_btf_obj_id. If it is an issue, maybe just hold the module refcount also.
For struct_ops map, it attaches the struct_ops prog to the module, so the module refcount is held. It should be fine.
Re: [PATCH] bpf: Release module BTF IDR before module unload
Quoted 4 lines · Martin KaFai Lau
On 3/11/26 6:03 AM, Kumar Kartikeya Dwivedi wrote:
Commit message is missing?...
Quoted 45 lines
Quoted 29 lines
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- kernel/bpf/btf.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)kernel/bpf/btf.c +8
-1@@ -1883,10 +1883,17 @@ void btf_get(struct btf *btf)refcount_inc (&btf->refcnt); } +static bool btf_is_module (const struct btf *btf); +void btf_put (struct btf *btf) {if (btf &&refcount_dec_and_test (&btf->refcnt)) { -btf_free_id (btf); +/* +* IDR for module BTFs is freed when module's BTF reference is +* dropped, which may happen before the final put. +*/ +if (!btf_is_module (btf)) +btf_free_id (btf);call_rcu (&btf->rcu, btf_free_rcu); } }kernel/bpf/btf.c static int btf_module_notify(struct notifier_block *nb, unsigned long op, · line 8399
Quoted 10 lines
kernel/bpf/btf.c +6
-0@@ -8383,6 +8390,12 @@ static int btf_module_notify(struct notifier_block *nb, unsigned long op,if (btf_mod->module != module)continue ; +/* +* For modules, we do the freeing of BTF IDR as soon as +* module goes away to disable BTF discovery, since the +* btf_try_get_module() on such BTFs will fail. +*/ +btf_free_id (btf_mod->btf);list_del (&btf_mod->list);The change makes sense.
I looked up a bit because I am interested in who will use the module's
btf->idwithout holding the module's refcount. fwiw, here is what I read.
Quoted 6 lines
For non
struct_opsprog, the module refcount is held inprog->aux->modafter commit31bf1dbccfb0, so should be fine on thebpf_trampoline_compute_key()andbpf_prog_get_info_by_fd().For
struct_opsprog, the module refcount is not held inprog->aux->mod. I don't think it has to since it is not directly attached to the module.
If we have something referencing a module, we should probably get and keep refcnt on it, no? Why would we have a pointer to a module we cannot be sure is still alive when we access it, sounds too error-prone to me.
Quoted 7 lines
The only downside is the
bpf_prog_get_info_by_fd()may report an out-dated info.attach_btf_obj_id. If it is an issue, maybe just hold the module refcount also.For
struct_opsmap, it attaches thestruct_opsprog to the module, so the module refcount is held. It should be fine.
Martin KaFai Lau
reviewerRe: [PATCH] bpf: Release module BTF IDR before module unload
Quoted 11 lines · Andrii Nakryiko
Quoted 2 lines
For
struct_opsprog, the module refcount is not held inprog->aux->mod. I don't think it has to since it is not directly attached to the module.If we have something referencing a module, we should probably get and keep refcnt on it, no? Why would we have a pointer to a module we cannot be sure is still alive when we access it, sounds too error-prone to me.
Quoted 2 lines
For
struct_opsmap, it attaches thestruct_opsprog to the module, so the module refcount is held. It should be fine.
The module count is held but it is held by the struct_ops map. The struct_ops prog doesn't use the module and doesn't have a pointer to a module after it is loaded.
kernel test robot
build botRe: [PATCH] bpf: Release module BTF IDR before module unload
In reply to Kumar Kartikeya Dwivedikernel test robot noticed the following build errors:
Hi Kumar,
kernel test robot noticed the following build errors:
[auto build test ERROR on bpf-next/net]
[also build test ERROR on bpf-next/master bpf/master linus/master v6.16-rc1 next-20260311] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Kumar-Kartikeya-Dwivedi/bpf-Release-module-BTF-IDR-before-module-unload/20260311-211036 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git net patch link: https://lore.kernel.org/r/20260311130321.1055808-1-memxor%40gmail.com patch subject: [PATCH] bpf: Release module BTF IDR before module unload config: x86_64-rhel-9.4-ltp (https://download.01.org/0day-ci/archive/20260312/[email protected]/config) compiler: gcc-14 (Debian 14.2.0-19) 14.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260312/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> kernel/bpf/btf.c:1886:13: error: static declaration of 'btf_is_module' follows non-static declaration
1886 | static bool btf_is_module(const struct btf *btf);
| ^~~~~~~~~~~~~
In file included from include/linux/bpf.h:29,
from include/linux/bpf_verifier.h:7,
from kernel/bpf/btf.c:19:
include/linux/btf.h:219:6: note: previous declaration of 'btf_is_module' with type 'bool(const struct btf *)' {aka '_Bool(const struct btf *)'}
219 | bool btf_is_module(const struct btf *btf);
| ^~~~~~~~~~~~~vim +/btf_is_module +1886 kernel/bpf/btf.c
1885
> 1886 static bool btf_is_module(const struct btf *btf);
1887
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki