sched_ext: Work around pahole 1.32 dropping scx_bpf_task_set_lazy_resched() from BTF
Subsystems (3)
Tejun Heo
reviewerPATCH 1/1 sched_ext: Work around pahole 1.32 dropping scx_bpf_task_set_lazy_resched() from BTF
With clang and pahole 1.32, x86-64 vmlinux BTF lacks scx_bpf_task_set_lazy_resched() and sched_ext fails to initialize:
sched_ext: Failed to register kfunc sets (-22)
clang describes the lazy argument only after the prologue moves it into a callee-saved register, which pahole 1.32 rejects as not being the argument register. Released pahole is affected whatever the eventual fix. Keep the argument on the stack with barrier_data() so that its location names no register. OPTIMIZER_HIDE_VAR() would fix x86-64 but break arm64.
--- kernel/sched/ext/ext.c | 5 +++++ 1 file changed, 5 insertions(+)
kernel/sched/ext/ext.c +5 -0
@@ -10136,6 +10136,11 @@ __bpf_kfunc bool scx_bpf_task_set_lazy_r {struct scx_sched *sch; +/* +* clang's register locations for @lazy make pahole 1.32 drop this kfunc +* from BTF. Keep @lazy in memory so its location names no register. +*/ +barrier_data (&lazy);guard (rcu)(); sch =scx_prog_sched (aux);if (unlikely (!sch || !scx_task_on_sched (sch, p)))
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] sched: build failure due to misapplied patch in scx_bpf_task_set_dsq_vtime() --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] sched: build failure due to misapplied patch in scx_bpf_task_set_dsq_vtime() --
commit 16f6b2129331609ed5db39cd3ea7260f54a1df1f
Author: Tejun Heo <[email protected]>
sched_ext: Work around pahole 1.32 dropping scx_bpf_task_set_lazy_resched() from BTF
This commit works around an issue with pahole 1.32 by forcing the lazy argument onto the stack using barrier_data() so its location doesn't name a register.
kernel/sched/ext/ext.c __bpf_kfunc bool scx_bpf_task_set_dsq_vtime(struct task_struct *p, u64 vtime, · line 10126
Quoted 13 lines
kernel/sched/ext/ext.c +5
-0@@ -10119,6 +10119,11 @@ __bpf_kfunc bool scx_bpf_task_set_dsq_vtime(struct task_struct *p, u64 vtime, {struct scx_sched *sch; +/* +* clang's register locations for @lazy make pahole 1.32 drop this kfunc +* from BTF. Keep @lazy in memory so its location names no register. +*/ +barrier_data (&lazy);
[Severity: Low]
Will this cause a build failure? It looks like the lazy variable is not declared in scx_bpf_task_set_dsq_vtime().
Could it be that this patch was mistakenly applied to scx_bpf_task_set_dsq_vtime() instead of scx_bpf_task_set_lazy_resched()? They appear to share identical context lines, meaning the patch might have matched guard(rcu)(); followed by sch = scx_prog_sched(aux); in the wrong function.
Quoted 4 lines
kernel/sched/ext/ext.c +0
-0guard (rcu)(); sch =scx_prog_sched (aux);if (unlikely (!sch || !scx_task_on_sched (sch, p)))return false;
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Andrea Righi
reviewerHi Tejun,
Quoted 21 lines · Tejun Heo
With clang and pahole 1.32, x86-64 vmlinux BTF lacks
scx_bpf_task_set_lazy_resched()andsched_extfails to initialize:
sched_ext: Failed to register kfunc sets (-22)clang describes the lazy argument only after the prologue moves it into a callee-saved register, which pahole 1.32 rejects as not being the argument register. Released pahole is affected whatever the eventual fix. Keep the argument on the stack with
barrier_data()so that its location names no register.OPTIMIZER_HIDE_VAR()would fix x86-64 but break arm64.Fixes f8e5a4e3f3be ("sched_ext: Add lazy preemption support")Signed-off-by Tejun Heo <[email protected]>--- kernel/sched/ext/ext.c | 5 +++++ 1 file changed, 5 insertions(+)
kernel/sched/ext/ext.c __bpf_kfunc bool scx_bpf_task_set_lazy_r · line 10143
Quoted 8 lines
kernel/sched/ext/ext.c +5
-0@@ -10136,6 +10136,11 @@ __bpf_kfunc bool scx_bpf_task_set_lazy_r {struct scx_sched *sch; +/* +* clang's register locations for @lazy make pahole 1.32 drop this kfunc +* from BTF. Keep @lazy in memory so its location names no register. +*/ +barrier_data (&lazy);
First of all thanks for figuring this out, it was really annoying, I've been holding pahole to 1.31 because of this.
I also looked a bit and it seems to be triggered specifically by the bool type. This is with clang 22.1.8 on x86_64 without barrier_data and only changing the type:
bool movl %esi, %ebp -> dropped by pahole u32 movl %esi, %ebp -> ok u64 movq %rsi, %r15 -> ok
With u32 the generated code is identical to the bool case, but clang keeps the RSI entry range, so pahole is happy.
Should we just change lazy to u32 (or use a u64 flags) instead of using barrier_data()?
Thanks,
-Andrea
Quoted 3 lines
kernel/sched/ext/ext.c +0
-0guard (rcu)(); sch =scx_prog_sched (aux);if (unlikely (!sch || !scx_task_on_sched (sch, p)))
Tejun Heo
reviewerQuoted 11 lines · Andrea Righi
First of all thanks for figuring this out, it was really annoying, I've been holding pahole to 1.31 because of this.
I also looked a bit and it seems to be triggered specifically by the bool type. This is with clang 22.1.8 on
x86_64withoutbarrier_dataand only changing the type:bool movl %esi, %ebp -> dropped by pahole u32 movl %esi, %ebp -> ok u64 movq %rsi, %r15 -> ok
Quoted 5 lines
With u32 the generated code is identical to the bool case, but clang keeps the RSI entry range, so pahole is happy.
Should we just change lazy to u32 (or use a u64 flags) instead of using
barrier_data()?
I mean, bool is the right and I'm hoping clang can be fixed and the workaround seems to work, so keep it as-is unless there are other reasons to change it (e.g. this becomes a wider interface taking u64 flags or sth like that)?
Thanks.
Signature · 1 line
tejun
Quoted 15 lines · Tejun Heo
Quoted 14 lines · Andrea Righi
First of all thanks for figuring this out, it was really annoying, I've been holding pahole to 1.31 because of this.
I also looked a bit and it seems to be triggered specifically by the bool type. This is with clang 22.1.8 on
x86_64withoutbarrier_dataand only changing the type:bool movl %esi, %ebp -> dropped by pahole u32 movl %esi, %ebp -> ok u64 movq %rsi, %r15 -> okWith u32 the generated code is identical to the bool case, but clang keeps the RSI entry range, so pahole is happy.
Quoted 7 lines
Quoted 2 lines
Should we just change lazy to u32 (or use a u64 flags) instead of using
barrier_data()?I mean, bool is the right and I'm hoping clang can be fixed and the workaround seems to work, so keep it as-is unless there are other reasons to change it (e.g. this becomes a wider interface taking u64 flags or sth like that)?
Yeah, at the moment I don't see any valid reason to change lazy to a u64 flags. And it's probably better to introduce a workaround in the code rather than a workaround in the ABI, so let's keep barrier_data().
Thanks,
-Andrea
Applied to sched_ext/for-7.4.
Thanks.
--
tejun
[email protected]
AI reviewDoes the Fixes tag reference a commit that exists in this branch's history? The tag points to f8e5a4e3f3be, but if that commit is not an ancestor of this commit, the Fixes tag would be referencing a…
Quoted 7 lines
sched_ext: Work around pahole 1.32 droppingscx_bpf_task_set_lazy_resched()from BTFWith clang and pahole 1.32, x86-64 vmlinux BTF lacks
scx_bpf_task_set_lazy_resched()andsched_extfails to initialize:
sched_ext: Failed to register kfunc sets (-22)
Quoted 7 lines
clang describes the lazy argument only after the prologue moves it into a callee-saved register, which pahole 1.32 rejects as not being the argument register. Released pahole is affected whatever the eventual fix. Keep the argument on the stack with
barrier_data()so that its location names no register.OPTIMIZER_HIDE_VAR()would fix x86-64 but break arm64.Fixes f8e5a4e3f3be ("sched_ext: Add lazy preemption support")
Does the Fixes tag reference a commit that exists in this branch's history? The tag points to f8e5a4e3f3be, but if that commit is not an ancestor of this commit, the Fixes tag would be referencing a commit from a different tree or branch.
Can you verify that f8e5a4e3f3be is reachable from this commit's history, or clarify which commit in this branch originally introduced the lazy preemption support that needs the workaround?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/36651688234