dwarf_loader: Trust the entry value register over the first location entry
Yonghong Song
reviewerPATCH 1/2 dwarf_loader: Trust the entry value register over the first location entry
A parameter's location list describes where the parameter lives from each entry's start address on, and a producer may leave the entry PC out of the list entirely, as clang does when the parameter is moved into a callee-saved register during the prologue. Here is @lazy, the second parameter of a kfunc in a clang 21.1.8 x86-64 vmlinux:
0x00dcf39e: DW_TAG_subprogram
DW_AT_low_pc (0xffffffff81553d70)
DW_AT_high_pc (0xffffffff81553ee3)
...
DW_AT_name ("scx_bpf_task_set_lazy_resched")
DW_AT_decl_file ("kernel/sched/ext/ext.c")
... 0x00dcf3c3: DW_TAG_formal_parameter
DW_AT_location (indexed (0x1491) loclist = 0x001e825f:
[0xffffffff81553d8b, 0xffffffff81553e5f): DW_OP_reg6 RBP
[0xffffffff81553e5f, 0xffffffff81553ec8): DW_OP_entry_value(DW_OP_reg4 RSI), DW_OP_stack_value
[0xffffffff81553ec8, 0xffffffff81553eca): DW_OP_reg6 RBP
[0xffffffff81553eca, 0xffffffff81553ecc): DW_OP_entry_value(DW_OP_reg4 RSI), DW_OP_stack_value
[0xffffffff81553ecc, 0xffffffff81553ee3): DW_OP_reg6 RBP)
DW_AT_name ("lazy")
DW_AT_decl_file ("kernel/sched/ext/ext.c")
DW_AT_decl_line (9747)
DW_AT_type (0x00d59515 "bool")@lazy arrives in RSI, but the function starts at 0xffffffff81553d70 while the list starts at 0xffffffff81553d8b: the range covering the function entry is missing and the first entry present names RBP, the register @lazy was moved to. parameter__decode_location() took the register from that first entry and only consulted DW_OP_entry_value when no register had been found yet, so loc_reg ended up as RBP, the parameter looked like it was in an unexpected register and the whole function was dropped from BTF:
scx_bpf_task_set_lazy_resched : skipping BTF encoding of function due to unexpected register usage for parameter
which in turn breaks the kernel build for a kfunc, the failure Tejun Heo reported:
WARN: resolve_btfids: no BTF func for kfunc scx_bpf_task_set_lazy_resched in scx_kfunc_ids_any
WARN: resolve_btfids: unresolved symbol scx_bpf_task_set_lazy_resched
DW_OP_entry_value(DW_OP_regN) says the parameter still holds the value regN had on entry to the function, so regN is by definition the register the parameter was passed in. That is better evidence than the first location list entry, so prefer it. Parameters described by DW_OP_piece keep the registers the pieces name, since a single entry value register cannot describe a multi-register aggregate.
On the vmlinux above this encodes 34 more functions, among them the kfunc, and drops none.
--- dwarf_loader.c | 34 +++++++++++++++++++++++++++------- 1 file changed, 27 insertions(+), 7 deletions(-)
dwarf_loader.c +27 -7
@@ -1736,6 +1736,11 @@ static void parameter__set_loc_reg(struct parameter *parm, int reg) parm->loc_reg = reg; } +static bool parameter__has_piece_info (const struct parameter *parm) +{ +return parm->first_reg_fields || parm->second_reg_fields; +} +static void parameter__set_field_bit (unsigned long *fields,int byte_offset) {if (byte_offset >=0 && byte_offset < (int )(sizeof (*fields) *8 ))
@@ -1848,6 +1853,7 @@ static void parameter__decode_location(Dwarf_Attribute *attr, struct conf_load * struct cu *cu, Dwarf_Die *die, struct parameter *parm) { + int entry_value_reg = PARAMETER_UNKNOWN_REG;Dwarf_Addr base, start, end;Dwarf_Op *expr, *entry_ops;Dwarf_Attribute entry_attr;
@@ -1893,15 +1899,34 @@ static void parameter__decode_location(Dwarf_Attribute *attr, struct conf_load *break ;case DW_OP_entry_value:case DW_OP_GNU_entry_value: -if (dwarf_getlocation_attr (attr, expr, &entry_attr) ==0 && +if (entry_value_reg ==PARAMETER_UNKNOWN_REG && +dwarf_getlocation_attr (attr, expr, &entry_attr) ==0 &&dwarf_getlocation (&entry_attr, &entry_ops, &entry_len) ==0 && entry_len ==1 &&dwarf_op__is_reg (entry_ops->atom)) -parameter__set_loc_reg (parm, entry_ops->atom); + entry_value_reg = entry_ops->atom;break ; } }libdw__lock_unlock (); +/* +* DW_OP_entry_value(DW_OP_regN) says the parameter still holds the +* value regN had on entry to the function, so regN is the register the +* parameter was passed in. Prefer it over the register named by the +* first location list entry: that entry only describes where the +* parameter lives from its own start address on, and producers do omit +* the entry PC from the list, as clang does when the parameter is +* moved into a callee-saved register during the prologue: the first +* entry then names that register instead of the argument register, +* making the whole function look like it uses unexpected registers. +* +* Parameters described by pieces keep the register(s) the pieces name; +* a single entry value register cannot describe a multi-register +* aggregate. +*/ +if (entry_value_reg !=PARAMETER_UNKNOWN_REG && !parameter__has_piece_info (parm)) + parm->loc_reg = entry_value_reg; +parameter__finish_piece_decode (parm, die, conf, cu); }
@@ -3692,11 +3717,6 @@ static int parameter__align_reg_idx(const struct parameter *parm, int reg_idx,return (reg_idx + align -1 ) & ~(align -1 ); } -static bool parameter__has_piece_info (const struct parameter *parm) -{ -return parm->first_reg_fields || parm->second_reg_fields; -} -static bool parameter__uses_full_aggregate (const struct parameter *parm) {return parm->first_reg_fields && parm->second_reg_fields;
Signature · 1 line
2.53.0-Meta
Yonghong Song
reviewerPATCH 2/2 tests: Add test for parameters described by entry values
Cover the case where clang leaves the entry PC out of a parameter's location list because the parameter is moved into a callee-saved register during the prologue. f_lazy() reproduces the shape found in the kernel:
@lazy arrives in RSI, the first location list entry starts past the function entry and names the callee-saved register, and only the DW_OP_entry_value entries name RSI.
Without the entry value being preferred, @lazy looks like it is in an unexpected register and f_lazy() is dropped from BTF altogether.
The test results on x86_64:
$ VERBOSE=1 ./clang_parm_entry_value.sh
Validation of BTF encoding of parameters described by entry values.
BTF: bool f_lazy(struct task * p, bool lazy, const struct aux * aux);
Test ./clang_parm_entry_value.sh passed--- tests/clang_parm_entry_value.sh | 84 +++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100755 tests/clang_parm_entry_value.sh
tests/clang_parm_entry_value.sh new file +84 -0
@@ -0,0 +1,84 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0-only + +source "$(dirname "$0")/test_lib.sh" + +outdir=$(make_tmpdir) + +# Comment this out to save test data. +trap cleanup EXIT + +title_log "Validation of BTF encoding of parameters described by entry values." + +entry_value="${outdir}/entry_value" +CC=$(which clang 2>/dev/null) + +if [[ -z "$CC" ]]; then + info_log "skip: clang not available" + test_skip +fi + +arch=$(uname -m) +if [[ "$arch" != "x86_64" ]]; then + info_log "skip: test is x86_64 only, running on $arch" + test_skip +fi + +# clang can leave the entry PC out of a parameter's location list when the +# parameter is moved into a callee-saved register during the prologue. The +# first location list entry then names that callee-saved register rather than +# the ABI argument register, and only the DW_OP_entry_value entries tell which +# argument register the parameter really arrived in. pahole has to use the +# entry value, otherwise the parameter looks like it is in an unexpected +# register and the whole function is dropped from BTF. +cat > ${entry_value}.c << EOF +typedef _Bool bool; +struct sched; +struct aux; +struct task { long a; bool lazy; }; + +extern struct sched *get_sched(const struct aux *aux); +extern void rcu_lock(void); +extern void rcu_unlock(void); +extern int on_sched(struct sched *s, struct task *p); + +__attribute__((noinline)) bool f_lazy(struct task *p, bool lazy, const struct aux *aux) +{ + struct sched *sch; + + rcu_lock(); + sch = get_sched(aux); + if (__builtin_expect(!(sch && on_sched(sch, p)), 0)) { + rcu_unlock(); + return 0; + } + __atomic_store_n(&p->lazy, lazy, __ATOMIC_RELAXED); + rcu_unlock(); + return 1; +} + +bool (*keep[])() = { (bool(*)())f_lazy }; +EOF + +${CC} -g -O2 -c -o ${entry_value}.o ${entry_value}.c 2>/dev/null +if [[ $? -ne 0 ]]; then + info_log "skip: clang could not compile ${entry_value}.c" + test_skip +fi + +LLVM_OBJCOPY=objcopy pahole -J --btf_features=consistent_func ${entry_value}.o +if [[ $? -ne 0 ]]; then + error_log "Could not encode BTF for ${entry_value}.o" + test_fail +fi + +encoded=$(pfunct --all --format_path=btf ${entry_value}.o | grep " f_lazy(") +if [[ -n "$VERBOSE" ]]; then + printf " BTF: %s\n" "$encoded" +fi +if [[ -z "$encoded" ]]; then + error_log "f_lazy() is missing from BTF" + test_fail +fi + +test_pass
Signature · 1 line
2.53.0-Meta
Quoted 4 lines · Yonghong Song
DW_OP_entry_value(DW_OP_regN)says the parameter still holds the value regN had on entry to the function, so regN is by definition the register the parameter was passed in. That is better evidence than the first location list entry, so prefer it.
my bot is saying that this is not quite the case.
gcc 13 -O2 for
long f6(long a, long b)
{
b = a;
ext();
ext();
return 0;
}describes 'b' as:
[0x40, 0x44): DW_OP_reg4 RSI
[0x44, 0x4c): DW_OP_reg5 RDI
[0x4c, 0x59): DW_OP_entry_value(DW_OP_reg5 RDI), DW_OP_stack_value
pahole encodes f6() today. With this patch loc_reg of 'b' becomes RDI, RSI is expected, and f6() is dropped.
dwarf_loader.c static void parameter__decode_location(Dwarf_Attribute *attr, struct conf_load * · line 1928
Quoted 2 lines
dwarf_loader.c +2
-0+if (entry_value_reg !=PARAMETER_UNKNOWN_REG && !parameter__has_piece_info (parm)) + parm->loc_reg = entry_value_reg;
This fixes only the functions where clang happened to emit an entry value. clang -O2 for
bool g_lazy(struct task *p, bool lazy)
{
rcu_lock();
__atomic_store_n(&p->lazy, lazy, __ATOMIC_RELAXED);
rcu_unlock();
return lazy;
}describes 'lazy' as:
[0x09, 0x1e): DW_OP_reg3 RBX
[0x1e, 0x21): DW_OP_reg0 RAX
low_pc is 0. There is no entry value, so g_lazy() is still dropped. With 'int lazy' the list starts at low_pc with RSI. Looks like clang loses the entry range for every bool arg.
Yonghong Song
reviewerQuoted 17 lines · Alexei Starovoitov
Quoted 4 lines · Yonghong Song
DW_OP_entry_value(DW_OP_regN)says the parameter still holds the value regN had on entry to the function, so regN is by definition the register the parameter was passed in. That is better evidence than the first location list entry, so prefer it.my bot is saying that this is not quite the case.
gcc 13 -O2 for
long
f6(long a, long b){ b = a; ext(); ext(); return 0; }
Quoted 4 lines
describes 'b' as:
[0x40, 0x44): DW_OP_reg4 RSI [0x44, 0x4c): DW_OP_reg5 RDI [0x4c, 0x59): DW_OP_entry_value(DW_OP_reg5 RDI), DW_OP_stack_value
I tried with the above example.
$ cat t1.c
void ext(void);
long f6(long a, long b)
{
b = a;
ext();
ext();
return 0;
}$ gcc -O2 -c -g t1.c. # gcc14 $ llvm-dwarfdump t1.o ...
0x00000053: DW_TAG_formal_parameter
DW_AT_name ("a")
DW_AT_decl_file ("/home/yhs/tmp/t1.c")
DW_AT_decl_line (2)
DW_AT_decl_column (14)
DW_AT_type (0x0000008e "long int")
DW_AT_location (0x00000010:
[0x0000000000000000, 0x0000000000000008): DW_OP_reg5 RDI
[0x0000000000000008, 0x0000000000000015): DW_OP_entry_value(DW_OP_reg5 RDI), DW_OP_stack_value)
DW_AT_GNU_locviews (0x0000000c)0x00000063: DW_TAG_formal_parameter
DW_AT_name ("b")
DW_AT_decl_file ("/home/yhs/tmp/t1.c")
DW_AT_decl_line (2)
DW_AT_decl_column (22)
DW_AT_type (0x0000008e "long int")
DW_AT_location (0x0000002d:
[0x0000000000000000, 0x0000000000000000): DW_OP_reg4 RSI
[0x0000000000000000, 0x0000000000000008): DW_OP_reg5 RDI
[0x0000000000000008, 0x0000000000000015): DW_OP_entry_value(DW_OP_reg5 RDI), DW_OP_stack_value)
DW_AT_GNU_locviews (0x00000027)Maybe we need to poke into locations to get precise parameter register. For example, for argument 'b', we have
[0x0000000000000000, 0x0000000000000000): DW_OP_reg4 RSI ... [0x0000000000000008, 0x0000000000000015): DW_OP_entry_value(DW_OP_reg5 RDI), DW_OP_stack_value)
RSI has location starts from 0x0, RDI starts from 0x8 although it has DW_OP_entry_value.
But this makes things complicated, and there is no garantee across different compilers due to code gen.
Quoted 3 lines
pahole encodes
f6()today. With this patchloc_regof 'b' becomes RDI, RSI is expected, andf6()is dropped.
Okay, that means my patch actually not working.
Quoted 14 lines
dwarf_loader.c static void parameter__decode_location(Dwarf_Attribute *attr, struct conf_load * · line 1928
Quoted 3 lines
dwarf_loader.c +2
-0+if (entry_value_reg !=PARAMETER_UNKNOWN_REG && !parameter__has_piece_info (parm)) + parm->loc_reg = entry_value_reg;This fixes only the functions where clang happened to emit an entry value. clang -O2 for
bool
g_lazy(struct task *p, bool lazy){ rcu_lock(); __atomic_store_n(&p->lazy, lazy, __ATOMIC_RELAXED); rcu_unlock(); return lazy; }
Quoted 7 lines
describes 'lazy' as:
[0x09, 0x1e): DW_OP_reg3 RBX [0x1e, 0x21): DW_OP_reg0 RAX
low_pcis 0. There is no entry value, sog_lazy()is still dropped. With 'int lazy' the list starts atlow_pcwith RSI. Looks like clang loses the entry range for every bool arg.
In some other cases, clang may generate even more complicated dwarf for bool parameters.
One possible workaround is change 'lazy' type from 'bool' to say 'int'. As Alexei mentioned in the above, clang didn't handle 'bool' well in debug info.
Hello, Yonghong.
Quoted 3 lines · Yonghong Song
One possible workaround is change 'lazy' type from 'bool' to say 'int'. As Alexei mentioned in the above, clang didn't handle 'bool' well in debug info.
OPTIMIZER_HIDE_VAR(lazy) at the top of the function works too and I can carry that from the kernel side.
Is this a clang bug that should be fixed? @lazy sits untouched in RSI until the prologue moves it, and clang describes an int parameter there but not a bool one.
Thanks.
--
tejun
Quoted 9 lines · Yonghong Song
Maybe we need to poke into locations to get precise parameter register. For example, for argument 'b', we have
[0x0000000000000000, 0x0000000000000000): DW_OP_reg4 RSI ... [0x0000000000000008, 0x0000000000000015): DW_OP_entry_value(DW_OP_reg5 RDI), DW_OP_stack_value)RSI has location starts from
0x0, RDI starts from0x8although it hasDW_OP_entry_value.But this makes things complicated, and there is no garantee across different compilers due to code gen.
I don't think it has to be complicated.
In your gcc 14 dump the first entry of 'b' starts at low_pc and it's RSI. In scx_bpf_task_set_lazy_resched() low_pc is ...d70 and the first entry of 'lazy' starts at ...d8b. RBP there says nothing about the register 'lazy' was passed in.parameter__decode_location() already has 'start' of every entry. Can it take the register of the first entry only when that entry starts at low_pc and use the entry value otherwise?
There is no guarantee, right, but when the guess is wrong the function is dropped. It's not encoded with a wrong signature.
Quoted 3 lines
One possible workaround is change 'lazy' type from 'bool' to say 'int'. As Alexei mentioned in the above, clang didn't handle 'bool' well in debug info.
That works for this kfunc and for pahole that is already released, but every other function with a bool arg is still dropped. Can clang be fixed to keep the entry range?
Yonghong Song
reviewerQuoted 10 lines · Alexei Starovoitov
Quoted 9 lines · Yonghong Song
Maybe we need to poke into locations to get precise parameter register. For example, for argument 'b', we have
[0x0000000000000000, 0x0000000000000000): DW_OP_reg4 RSI ... [0x0000000000000008, 0x0000000000000015): DW_OP_entry_value(DW_OP_reg5 RDI), DW_OP_stack_value)RSI has location starts from
0x0, RDI starts from0x8although it hasDW_OP_entry_value.But this makes things complicated, and there is no garantee across different compilers due to code gen.
Quoted 8 lines
I don't think it has to be complicated.
In your gcc 14 dump the first entry of 'b' starts atlow_pcand it's RSI. Inscx_bpf_task_set_lazy_resched()low_pcis ...d70 and the first entry of 'lazy' starts at ...d8b. RBP there says nothing about the register 'lazy' was passed in.parameter__decode_location()already has 'start' of every entry. Can it take the register of the first entry only when that entry starts atlow_pcand use the entry value otherwise?
Yes, this is what I thought as well.
Quoted 3 lines
There is no guarantee, right, but when the guess is wrong the function is dropped. It's not encoded with a wrong signature.
Okay, no guarantee, but it should be an improvement.
Quoted 7 lines
Quoted 4 lines
One possible workaround is change 'lazy' type from 'bool' to say 'int'. As Alexei mentioned in the above, clang didn't handle 'bool' well in debug info.
That works for this kfunc and for pahole that is already released, but every other function with a bool arg is still dropped. Can clang be fixed to keep the entry range?
Let me investigate this. Probably will be slow a little bit due lpc and some other events.