Misc BPF fixes from sashiko findings
Daniel Borkmann
maintainercover Misc BPF fixes from sashiko findings
Addressing some misc/random findings from sashiko that are worth fixing which popped up as pre-existing issues while working on the signed BPF loader series.
Daniel Borkmann (4):
bpf: Fix vmlinux BTF prep race in bpf_get_btf_vmlinux
bpf: Give vmlinux BTF init its own mutex
bpf: Account insn_aux_data allocation in bpf_check
bpf: Account scratch buffer in bpf_prog_calc_tag
kernel/bpf/btf.c | 2 +- kernel/bpf/core.c | 2 +- kernel/bpf/verifier.c | 30 ++++++++++++++++++++++-------- 3 files changed, 24 insertions(+), 10 deletions(-)
Signature · 1 line
2.43.0
Daniel Borkmann
maintainerPATCH 1/4 bpf: Fix vmlinux BTF prep race in bpf_get_btf_vmlinux
bpf_get_btf_vmlinux() lazily parses the vmlinux BTF under the bpf_verifier_lock, but publishes the result through a plain store and re-checks it through a plain lockless load. Nothing orders the stores initializing the struct btf inside btf_parse_vmlinux() against the store publishing the pointer: On a weakly ordered arch, a concurrent first-time caller taking the lockless fast path could in principle observe the pointer before the parsed contents are visible. The mutex_unlock() does not help such a reader given it only synchronizes with a later acquisition of the same lock. Thus, publish the pointer with smp_store_release() and read it on the fast path with smp_load_acquire().
Acquire semantics are needed rather than a dependency-ordered READ_ONCE(): btf_parse_vmlinux() also populates globals outside the returned object (e.g. bpf_ctx_convert.t). An address dependency would only order accesses performed through the pointer and not cover other globals.
--- kernel/bpf/verifier.c | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-)
kernel/bpf/verifier.c +16 -4
@@ -19559,13 +19559,25 @@ int bpf_check_attach_btf_id_multi(struct btf *btf, struct bpf_prog *prog, u32 btstruct btf *bpf_get_btf_vmlinux (void ) { -if (!btf_vmlinux &&IS_ENABLED (CONFIG_DEBUG_INFO_BTF )) { +/* Pairs with the smp_store_release() on the parse path below. */ +struct btf *btf =smp_load_acquire (&btf_vmlinux); + +if (!btf &&IS_ENABLED (CONFIG_DEBUG_INFO_BTF )) {mutex_lock (&bpf_verifier_lock); -if (!btf_vmlinux) - btf_vmlinux =btf_parse_vmlinux (); + btf = btf_vmlinux; +if (!btf) { + btf =btf_parse_vmlinux (); +/* +* Order the parsed BTF contents and the globals the +* parse populated (e.g. bpf_ctx_convert.t) before +* the pointer publication. Pairs with the acquire +* on the lockless fast path above. +*/ +smp_store_release (&btf_vmlinux, btf); + }mutex_unlock (&bpf_verifier_lock); } -return btf_vmlinux; +return btf; }/*
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH bpf-next 1/4] bpf: Fix vmlinux BTF prep race in bpf_get_btf_vmlinux
This looks like a fix for a real memory-ordering bug on weakly ordered architectures. Should it carry a Fixes: tag?
Quoted 14 lines
bpf: Fix vmlinux BTF prep race in
bpf_get_btf_vmlinux
bpf_get_btf_vmlinux()lazily parses the vmlinux BTF under thebpf_verifier_lock, but publishes the result through a plain store and re-checks it through a plain lockless load. Nothing orders the stores initializing the struct btf insidebtf_parse_vmlinux()against the store publishing the pointer: On a weakly ordered arch, a concurrent first-time caller taking the lockless fast path could in principle observe the pointer before the parsed contents are visible. Themutex_unlock()does not help such a reader given it only synchronizes with a later acquisition of the same lock. Thus, publish the pointer withsmp_store_release()and read it on the fast path withsmp_load_acquire().
Quoted 7 lines
Acquire semantics are needed rather than a dependency-ordered
READ_ONCE():btf_parse_vmlinux()also populates globals outside the returned object (e.g. bpf_ctx_convert.t). An address dependency would only order accesses performed through the pointer and not cover other globals.Signed-off-by Daniel Borkmann <[email protected]>
This looks like a fix for a real memory-ordering bug on weakly ordered architectures. Should it carry a Fixes: tag?
The plain store publishing the pointer together with the plain lockless load appears to go back to the introduction of the in-kernel BTF parsing, so a candidate would be:
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/28977272970
Daniel Borkmann
maintainerPATCH 2/4 bpf: Give vmlinux BTF init its own mutex
In reply to Daniel Borkmannbpf_get_btf_vmlinux() serializes the lazy vmlinux BTF parse with bpf_verifier_lock, the same mutex bpf_check() holds across the whole verification of an unprivileged program (if enabled; it's disabled by default). The latter can potentially stall the mutex holder for a long time (e.g. via userfaultfd), and therefore block first-time bpf_get_btf_vmlinux() caller from any context, including privileged program loads.
Give the vmlinux BTF initialization a dedicated btf_vmlinux_lock so it is independent of the unprivileged verification mutex. The parse only needs mutual exclusion against itself.
--- kernel/bpf/btf.c | 2 +- kernel/bpf/verifier.c | 7 ++++--- 2 files changed, 5 insertions(+), 4 deletions(-)
kernel/bpf/btf.c +1 -1
@@ -6451,7 +6451,7 @@ struct btf *btf_parse_vmlinux(void)if (IS_ERR (btf)) goto err_out; -/* btf_parse_vmlinux() runs under bpf_verifier_lock */ +/* btf_parse_vmlinux() runs under btf_vmlinux_lock */ bpf_ctx_convert.t =btf_type_by_id (btf, bpf_ctx_convert_btf_id[0 ]); err =btf_alloc_id (btf);if (err) {
kernel/bpf/verifier.c +4 -3
@@ -324,6 +324,7 @@ static const char *btf_type_name(const struct btf *btf, u32 id) }static DEFINE_MUTEX (bpf_verifier_lock); +static DEFINE_MUTEX (btf_vmlinux_lock);static DEFINE_MUTEX (bpf_percpu_ma_lock); __printf(2, 3) static void verbose(void *private_data, const char *fmt, ...)
@@ -19563,7 +19564,7 @@ struct btf *bpf_get_btf_vmlinux(void)struct btf *btf =smp_load_acquire (&btf_vmlinux);if (!btf &&IS_ENABLED (CONFIG_DEBUG_INFO_BTF )) { -mutex_lock (&bpf_verifier_lock); +mutex_lock (&btf_vmlinux_lock); btf = btf_vmlinux;if (!btf) { btf =btf_parse_vmlinux ();
@@ -19575,7 +19576,7 @@ struct btf *bpf_get_btf_vmlinux(void)*/ smp_store_release (&btf_vmlinux, btf); } -mutex_unlock (&bpf_verifier_lock); +mutex_unlock (&btf_vmlinux_lock); }return btf; }
@@ -20089,7 +20090,7 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr,bpf_get_btf_vmlinux (); -/* grab the mutex to protect few globals used by verifier */ +/* Serialize verification of unprivileged programs. */ if (!is_priv)mutex_lock (&bpf_verifier_lock);
Signature · 1 line
2.43.0
Daniel Borkmann
maintainerPATCH 3/4 bpf: Account insn_aux_data allocation in bpf_check
In reply to Daniel BorkmannThe insn_aux_data array is allocated with a plain vzalloc(), while every other allocation scoped to the verification - verifier states, explored states, the cfg/scc arrays, liveness masks, jump history - is charged to the loader's memcg via GFP_KERNEL_ACCOUNT.
At 136 bytes per instruction it is one of the largest verification-time buffers, in the range of ~130MB for a program at the 1M instruction limit (worst case), and it lives across the whole verification. The buffer is also inconsistent with itself: when instruction patching grows it, the vrealloc() in bpf_patch_insn_data() already passes GFP_KERNEL_ACCOUNT.
--- kernel/bpf/verifier.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/bpf/verifier.c +2 -1
@@ -20096,7 +20096,8 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr, bpfptr_t uattr, len = env->prog->len; env->insn_aux_data = -vzalloc (array_size (sizeof (struct bpf_insn_aux_data ), len)); +__vmalloc (array_size (sizeof (struct bpf_insn_aux_data ), len), +GFP_KERNEL_ACCOUNT | __GFP_ZERO); ret = -ENOMEM ;if (!env->insn_aux_data) goto skip_full_check;
Signature · 1 line
2.43.0
Daniel Borkmann
maintainerPATCH 4/4 bpf: Account scratch buffer in bpf_prog_calc_tag
In reply to Daniel Borkmannbpf_prog_calc_tag() copies the instructions into a plain vmalloc() scratch buffer to blind the map fds before hashing. The buffer scales with the program, up to ~8MB at the 1M instruction limit, and is allocated on every program load, but unlike the rest of the load-time scratch memory it is not charged to the loader's memcg. Use GFP_KERNEL_ACCOUNT to account it like the other allocations scoped to the verification/load.
--- kernel/bpf/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/bpf/core.c +1 -1
@@ -305,7 +305,7 @@ int bpf_prog_calc_tag(struct bpf_prog *fp)bool was_ld_map;u32 i; - dst =vmalloc (size); + dst =__vmalloc (size,GFP_KERNEL_ACCOUNT );if (!dst)return -ENOMEM ;
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH bpf-next 4/4] bpf: Account scratch buffer in bpf_prog_calc_tag
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [Medium] Missing memcg accounting for the temporary JIT scratch buffer rw_header. --
--- Patch [4]: [PATCH bpf-next 4/4] bpf: Account scratch buffer in bpf_prog_calc_tag --- commit 57c46cbb1e055c7c0a73414dd57ed04bc1cb8b73
Author: Daniel Borkmann <[email protected]>
bpf: Account scratch buffer in bpf_prog_calc_tag
This commit updates bpf_prog_calc_tag() to use GFP_KERNEL_ACCOUNT for its vmalloc scratch buffer. This ensures the temporary memory, which scales with the program size, is properly charged to the loader's memory cgroup.
kernel/bpf/core.c int bpf_prog_calc_tag(struct bpf_prog *fp) · line 308
Quoted 9 lines
kernel/bpf/core.c +1
-1@@ -305,7 +305,7 @@ int bpf_prog_calc_tag(struct bpf_prog *fp)bool was_ld_map;u32 i; - dst =vmalloc (size); + dst =__vmalloc (size,GFP_KERNEL_ACCOUNT );
[Severity: Medium]
This is a pre-existing issue, but since this patch correctly applies memcg accounting to load-time scratch memory, should we also account the temporary JIT scratch buffer?
Looking at bpf_jit_binary_pack_alloc() in the same file, rw_header scales with the program size but uses GFP_KERNEL instead of GFP_KERNEL_ACCOUNT:
kernel/bpf/core.c:bpf_jit_binary_pack_alloc() {
...
*rw_header = kvmalloc(size, GFP_KERNEL);
...
}Does this missing accounting allow a process to bypass its memcg limits by triggering concurrent JIT compilations?
Quoted 2 lines
kernel/bpf/core.c +0
-0if (!dst)return -ENOMEM ;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:
Quoted 11 lines · On Wed, 8 Jul 2026 23:15:33 +0200 you wrote:
Addressing some misc/random findings from sashiko that are worth fixing which popped up as pre-existing issues while working on the signed BPF loader series.
Daniel Borkmann (4):
bpf: Fix vmlinux BTF prep race inbpf_get_btf_vmlinux
bpf: Give vmlinux BTF init its own mutex
bpf: Accountinsn_aux_dataallocation inbpf_check
bpf: Account scratch buffer inbpf_prog_calc_tag[...]
Here is the summary with links:
- [bpf-next,1/4] bpf: Fix vmlinux BTF prep race in bpf_get_btf_vmlinux
https://git.kernel.org/bpf/bpf-next/c/92863e678070 - [bpf-next,2/4] bpf: Give vmlinux BTF init its own mutex https://git.kernel.org/bpf/bpf-next/c/5e5e94d87dea - [bpf-next,3/4] bpf: Account insn_aux_data allocation in bpf_check https://git.kernel.org/bpf/bpf-next/c/42560699a83d - [bpf-next,4/4] bpf: Account scratch buffer in bpf_prog_calc_tag https://git.kernel.org/bpf/bpf-next/c/ff755b600790
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html